Deep Generative Models

 The primal goal of generative models is learning a density estimator for a given data like sound, pictures and sentences.

 Thanks to the recently appeared bridge between deep learning and machine learning, many conventional machine learning problems have been solved with astonishing results.

 We have explored generative models in various aspects to get well generalized realistic samples from data.


Bayesian optimization

  Bayesian optimization is a method to optimize expensive black-box functions. It finds global optimum, evaluating the given domain space with Bayesian models. Applications of Bayesian optimization are hyperparameter optimization, architecture searching, experimental design, robotics, and etc.

  We are interested in solving the following issues in Bayesian optimization: (1) developing acquisition function for balancing exploration and exploitation, (2) transferring prior knowledge to new task for Bayesian optimization, and (3) optimizing automatically machine learning models.

  • Jungtaek Kim and Seungjin Choi (2018), "Clustering-guided GP-UCB for Bayesian optimization," in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP-2018), Calgary, Alberta, Canada, April 15-20, 2018.
  • Jungtaek Kim, Saehoon Kim, and Seungjin Choi (2017), "Learning to transfer initialization for Bayesian hyperparameter optimization," NIPS 2017 Workshop on Bayesian Optimization, December 9, 2017.

Meta-Learning

 Any learning algorithm depends on its underlying assumptions about data, otherwise known as its inductive bias.

 We are developing methods that can learn such assumptions from related datasets, thus creating learners that can (meta-) learn their own learning algorithms.


 Last updated: 2018/10/23


2019/11/27 PM 07:00
Overcoming catastrophic forgetting in neural networks. Kirkpatrick, James, et al. Proceedings of the national academy of sciences (2017): 201611835.
Presenter: Jinhwi Lee
Presentation Link

2019/11/20 PM 07:00
TBD
Presenter: Youngnam Kim
Presentation Link

2019/11/13 PM 07:00
Visualizing deep neural network decisions: Prediction difference analysis. (ICLR 2017) Zintgraf, Luisa M., et al.
Presenter: Nayeong Kim
Presentation Link

2019/10/23 PM 07:00
Self-Supervised Feature Learning by Learning to Spot Artifacts (CVPR 2018), Simon Jenni and Paolo Favaro
Presenter: Wonbin Kim
Presentation Link

2019/10/16 PM 07:00
Probabilistic Model-Agnostic Meta-Learning (arXiv 2018), Chelsea Finn, Kelvin Xu, and Sergey Levine
Presenter: Yeongseok Yoon
Presentation Link

2019/10/09 AM 11:00
Recent works on Batch Bayesian Optimization
Presenter: Jungtaek Kim
Presentation Link

2018/9/18 PM 7:00
Attention is all you need. Vaswani, Ashish, et al. Advances in Neural Information Processing Systems. 2017.
Presenter: Nayeong Kim
Presentation Link

2018/9/11 PM 7:00
GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training, (arXiv 2018), Samet Akcay et al
Presenter: Youngnam Kim
Presentation Link


2018/9/11 PM 7:00
Progress : Reimplementation – MAML and CNP (Conditional Neural Process)
Presenter: Youngseok Yoon
Presentation Link

2018/9/11 PM 7:00
Progress : Papers read in summer semester
Presenter: Nayeong Kim
Presentation Link

2018/9/11 PM 7:00
Progress : Towards developing improved GAN-based anomaly detection methods
Presenter: Youngnam Kim
Presentation Link


2018/6/12 PM 01:30
Anomaly Detection with Robust Deep Autoencoders (KDD, 2017), Chong Zhou, Randy C. Paffenroth
Presenter: Wonbin Kim
Presentation Link

2018/05/29 PM 07:00
Attention-based Deep Multiple Instance Learning (arXiv 2018), Maximilian Ilse, Jakub M. Tomczak, and Max Welling
Presenter: Yeongseok Yoon
Presentation Link

2018/05/15 PM 07:30
On First-Order Meta-Learning Algorithms (arXiv 2018), Alex Nichol, Joshua Achiam, and John Schulman
Additional Link
Presenter: Yoonho Lee
Presentation Link

2018/05/01 PM 07:00
Recurrent Neural Network Attention Mechanisms for Interpretable System Log Anomaly Detection. (arXiv preprint 2018), Brown, A., Tuor, A., Hutchinson, B., & Nichols, N.
Presenter: Nayeong Kim
Presentation Link

2018/04/16 PM 07:00
Implicit Causal Models for Genome-wide Association Studies (arXiv preprint 2017), Dustin Tran, David M. Blei.
Presentation Link
Presenter: Youngnam Kim

2018/04/03 PM 07:00
On Unifying Deep Generative Models(To appear in ICLR 2018), Zhiting Hu, Zichao Yang, Ruslan Salakhutdinov, Eric P. Xing.
Presentation Link
Presenter: Wonbin Kim

2018/03/27 PM 07:00
Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm(ICML, 2018), Chelsea Finn, Sergey Levine.
Presentation Link
Presenter: Yeongseok Yoon

2018/03/13 PM 04:00
New insights and perspectives on the natural gradient method(arXiv preprint 2014), James Martens.
Presentation Link
Presenter: Yoonho Lee

2018/02/27 PM 07:00
Talk : Learning Prior and Recent works
Presentation Link
Presenter: Suwon Suh

2018/02/20 PM 07:00
InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations(NIPS 2017), Li, Y., Song, J., & Ermon, S.
Presentation Link
Presenter: Nayeong Kim

2018/01/30 PM 07:00
Hybrid computing using a neural network with dynamic external memory(Nature, 2016), Alex Graves et al
Presentation Link
Presenter: Youngnam Kim

2018/01/23 PM 02:00
A Maximum Entropy Framework for Semisupervised and Active Learning With Unknown and Label-Scarce Classes (IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, VOL.28, NO.4, APRIL 2017), Zhicong Qiu, David J.Miller, and George Kesidis
Presentation Link
Presenter: Wonbin Kim

2018/01/09 PM 04:00
Stochastic Neural Networks for Hierarchical Reinforcement Learning (ICLR 2017), Carlos Florensa, Yan Duan, Pieter Abbeel
Presentation Link
Presenter: Youngseok Yoon

2017/12/19 PM 07:00
Meta Learning Shared Hierarchies (ICLR 2018 Review), Kevin Frans, Jonathan Ho, Xi Chen, Pieter Abbeel, John Schulman
Presentation Link
Presenter: Yoonho Lee

2017/11/28 PM 07:00
Reinforcement Learning with Unsupervised Auxiliary Tasks (ICLR 2017), Max Jaderberg, Volodymyr Mnih, Wojciech Marian Czarnecki, Joel Z Leibo, David Silver, Koray Kavukcuoglu
Presentation Link
Presenter: Youngnam Kim

2017/11/14 PM 07:00
Towards Open Set Deep Networks (CVPR 2017), Abhijit Bendale, Terrance Boult
Presentation Link
Presenter: Wonbin Kim

2017/11/07 PM 07:00
Input Switched Affine Networks: An RNN Architecture Designed for Interpretability (ICML 2017), Foerster, J.N., Gilmer, J., Sohl-Dickstein, J. Chorowski, J., Sussillo, D.
Presentation Link
Presenter: Nayeong Kim

2017/10/24 PM 07:00
Deep Reinforcement Learning with Double Q-Learning (AAAI 2016), Hado van Hasselt, Arthur Guez, David Silver
Presentation Link
Presenter: Youngseok Yoon

2017/10/17 PM 07:00
Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments (arXiv 2017), Maruan Al-Shedivat, Trapit Bansal, Yuri Burda, Ilya Sutskever, Igor Mordatch, Pieter Abbeel
Presentation Link
Presenter: Yoonho Lee

2017/05/23 PM 08:00
Variational Inference using Implicit Distributions (published in arXiv), Ferenc Huszár
Presentation Link
Presenter: Suwon Suh

2017/05/16 PM 08:00
Scalable Adaptive Stochastic Optimization Using Random Projections (NIPS 2016), Gabriel Krummenacher, Brian McWilliams, Yannic Kilcher, Joachim M. Buhmann, and Nicolai Meinshausen
Presenter: Saehoon Kim

2017/05/09 PM 08:00
Gradient Estimation Using Stochastic Computation Graphs (NIPS 2015), John Schulman, Nicolas Heess, Theophane Weber, and Pieter Abbeel
Presentation Link
Presenter: Yoonho Lee

2017/05/02 PM 08:00
Building Machines That Learn and Think Like People (published in arXiv), Brenden M. Lake, Tomer D. Ullman, Joshua B. Tenenbaum, and Samuel J. Gershman
Presenter: Aren Siekmeier

2017/04/24 PM 08:00
Improving Generative Adversarial Networks with Denoising Feature Matching (ICLR 2017), David Warde-Farley and Yoshua Bengio
Presentation Link
Presenter: Inhyuk Cho

2017/04/11 PM 08:00
Neural GPUs Learn Algorithms (ICLR 2016), Lukasz Kaiser and Ilya Sutskever
Presentation Link
Presenter: Minseop Park

2017/03/28 PM 08:00
Random Forest for the Contextual Bandit Problem (AISTATS 2016), Raphaël Féraud, Robin Allesiardo, Tanguy Urvoy, Fabrice Clérot
Presentation Link
Presenter: Jungtaek Kim

2017/03/21 PM 08:00
DelugeNets: Deep Networks with Massive and Flexible Cross-layer Information Inflows (published in arXiv), Jason Kuen, Xiangfei Kong, Gang Wang
Presentation Link
Presenter: Jiyuu Yi

2017/03/13 PM 09:00
Wasserstein GAN (published in arXiv), Martin Arjovsky, Soumith Chintala, Léon Bottou
Presentation Link
Presenter: Juho Lee

2017/02/13 AM 11:00
A Connection Between GANs, Inverse Reinforcement Learning, and Energy-Based Models (NIPS 2016 Workshop on Adversarial Training), Finn, Christiano, Abbeel and Levine
Presentation Link
Presenter: Suwon Suh

2017/01/23
Stacked Generative Adversarial Networks (under review for CVPR 2017), X. Huang et al.
Presentation Link
Presenter: Saehoon Kim



2016/11/29
Coupled Generative Adversarial Networks (NIPS 2016) Ming-Yu Liu and Oncel Tuzel
Presentation Link
Presenter: Inhuyk Cho

2016/11/22
Siamese Neural Networks for One-shot Image Recognition (ICML 2015 Deep Learning Workshop) Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov
Presentation Link
Presenter: Minseop Park

2016/11/14
Bayesian optimization under mixed constraints with a slack-variable augmented Lagrangian (NIPS 2016) Victor Picheny, Robert B. Gramacy, Stefan M. Wild, and Sebastien Le Digabel
Presentation Link
Presenter: Jungtaek Kim


2016/11/01
Composing graphical models with neural networks for structured representation and fast inference (NIPS 2016) M. J. Johnson, D. Duvenaud, A. B. Wiltschko, S. R. Datta and R. P. Adams
Presentation Link
Presenter: Juho Lee

2016/10/25
An Exploration of Parameter Redundancy in Deep Networks with Circulant Projections (ICCV 2015) Yu Cheng, Felix X. Yu, Rogerio S. Feris, Sanjiv Kumar, Alok Choudhary, Shih-Fu Chang
Presentation Link
Presenter: Saehoon Kim

2016/10/11
Dueling Network Architectures for Deep Reinforcement Learning (ICML 2016) Ziyu Wang, Tom Schaul, Matteo Hessel, Hado van Hasselt, Marc Lanctot, and Nando de Freitas
Presentation Link
Presenter: Yoonho Lee

2016/10/04
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets (NIPS 2016) Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel
Presentation Link
Presenter: Inhyuk Cho

2016/09/19
One-Shot Generaliztion in Deep Generative Models (ICML 2016) Danilo J. Rezende, Shakir, Mohamed, Ivo Danihelka, Karol Gregor, and Daan Wierstra
Presentation Link
Presenter: Minseop Park

2016/09/06
Robust Random Cut Forest Based Anomaly Detection On Streams (ICML 2016) Sudipto Guha, Nina Mishra, Gourav Roy, and Okke Schrijvers
Presentation Link
Presenter: Jungtaek Kim

2016/08/29
Deep Networks with Stochastic Depth Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Q. Weinberger
Presentation Link
Presenter: Jiyuu Yi

2016/08/22
The Variational Gaussian Process (ICLR 16) Dustin Tran, Rajesh Ranganath, and David M. Blei
Presentation Link
Presenter: Juho Lee

2016/08/09
Deep Kernel Learning (AISTATS 16) Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov, and Eric P. Xing
Presentation Link
Presenter: Saehoon Kim