CSED524/ITCE505 Probabilistic Graphical Models, Fall 2015


Announcements

References

Lectures

Dates and Titles Topics Lecture Slides Suggested Further Readings
Lecture 1
Introduction to PGM
  • Gentle introduction
  • Probability distributions


Lecture 2
Conditional Independence and Factorization

  • Conditional independence
  • Factorization


  • Chapter 2 in Jordan's PGM.
Lecture 3
Message Passing

  • Elimination algorithm
  • Sum product algorithm
  • Max product algorithm
  • Bethe free energy
  • Factor graphs


  • Chapter 3 and 4 in Jordan's PGM.
  • J. S. Yedidia, W. T. Freeman, and Y. Weiss,
    "Constructing free-energy approximations and
    generalized belief propagation algorithms,"
    IEEE Trans. Information Theory, vol. 51, no. 7, 2009.
Lecture 4
Junction tree algorithm

  • Junction tree algorithm


  • Chapter 17 in Jordan's PGM.
Lecture 5
Chow-Liu Tree

  • Chow-Liu tree


  • C. K. Chow and C. N. Liu,
    "Approximating discrete probability distributions with dependence trees,"
    IEEE Trans. Information Theory, vol. 14, no. 3, 1968.
Lecture 6
Density Estimation

  • Density estimation


  • ...
Lecture 7
Variational Inference

  • Variational Inference
  • Variational PCA
  • Variational MoG
  • Variational Bayesian Linear Regression
  • Variational Logistic Regression


  • Murphy 21
  • Bishop 10
  • H. Attias (1999),
    "Inferring parameters and structure of latent variable models by variational Bayes,"
    Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence, 1999.
  • C. Bishop (1999),
    "Bayesian PCA,"
    NIPS-1998.
  • M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul
    "An introduction to variational methods for graphical models,"
    In Learning in Graphical Models, Cambridge: MIT Press, 1999.
  • T. S. Jaakkola and M. I. Jordan (2000),
    "Bayesian logistic regression: a variational approach,"
    Statistics and Computing, vol. 10, 2000.
Lecture 8
Topic models

  • Probabilistic LSI
  • Latent Dirichlet allocation


  • T. Hofmann (1999),
    "Probabilistic latent semantic analysis,"
    UAI, 1999.
  • D. M. Blei, A. Ng, and M. I. Jordan (2003),
    "Latent Dirichlet allocation,"
    Journal of Machine Learning Research, 2003.
  • D. M. Blei and M. I. Jordan (2003),
    "Modeling annotated data,"
    SIGIR, 2003.
Lecture 9
Bayesian recommendation

  • Bayesian matrix factorization
  • Bayesian biomial mixture model for collaborative prediction


  • Yong-Deok Kim and Seungjin Choi (2014),
    "Bayesian binomial mixture model for collaborative prediction with non-random missing data,"
    RecSys-2014, 2014
  • Yong-Deok Kim and Seungjin Choi (2014),
    "Scalable variational Bayesian matrix factorization with side information,"
    AISTATS-2014, 2014.
  • Sunho Park, Yong-Deok Kim, and Seungjin Choi (2013),
    "Hierarchical Bayesian matrix factorization with side information,"
    IJCAI-2013, 2013.
  • Yong-Deok Kim and Seungjin Choi (2013),
    "Variational Bayesian view of weighted trace norm regularization for matrix factorization,"
    IEEE Signal Processing Letters, 2013.
Lecture 10
Sampling methods

  • Monte Carlo methods
  • Markov chain Monte Carlo
  • Sequential Monte Carlo


  • Chapter 11 in Bishop's PRML.
  • Chapter on sampling methods in Jordan's PGM.
  • C. Andrieu, N. De Freitas, A. Doucet, and M. I. Jordan (2003),
    "An introduction to MCMC for machine learning,"
    Machine Learning,
    vol. 50, pp. 5-43, 2003.
Lecture 11
Dirichlet processes

  • Dirichlet processes
  • DP mixture models


  • See the handout for the list of references
Lecture 12
Time series models

  • Linear dynamical systems (LDS)
  • Hidden Markov models (HMM)


  • ...

Homework Assignments

  • Hwk 1

  • Hwk 2

  • Hwk 3