EECE515 Machine Learning, Spring 2008


  • Announcements

  • Primary Textbook

    Pattern Recognition and Machine Learning (by C. Bishop, 2006)

  • Lectures

    Dates and Titles Topics Lecture Slides Suggested Further Readings
    Lecture 1
    Introduction to machine learning
    • Superised learning
    • Unsupervised learning
    • Probability primer


    Lecture 2
    Density estimation

    • Maximum likelihood estimation
    • MAP estimation
    • Bayesian estimation


    Lecture 3
    Clustering I

    • k-means clustering
    • Mixture of Gaussians (MoG)


    • Chapter 9.1 and 9.2 of Bishop's
    Lecture 4
    Expectation Maximization

    • Jensen's inequality
    • Information theory preliminaries
    • EM optimization
    • Generalized EM
    • Incremental EM
    • EM for exponetial families


    Lecture 5
    Clustering II

    • Spectral clustering
    • Nonnegative matrix factorization


    • J. Shi and J. Malik (2000),
      "Normalized Cuts and Image Segmentation",
      IEEE Trans. Pattern Analysis and Machine Intelligence,
      vol. 22, no. 8, pp. 888-905, 2000.
    • U. von Luxburg (2007),
      "A tutorial on spectral clustering,"
      Statistics and Computing,
      vol. 17, no. 4, pp. 395-416, 2007.
    • See my note on extremal properties of eigenvalues.
    • D. D. Lee and H. S. Seung (1999),
      "Learning the Parts of Objects by Non-negative Matrix Factorization",
      Nature,
      vol. 401, pp. 788-791, 1999.
    • C. Ding, T. Li, W. Peng, and H. Park (2006),
      "Orthogonal nonnegative matrix tri-factorizations for clustering,"
      KDD-2006.
    • A. Cichocki, H. Lee, Y.-D. Kim, and S. Choi (2008),
      "Nonnegative matrix factorization with alpha-divergence,"
      Pattern Recognition Letters, 2008 (in press).

    Lecture 5
    Latent variable models

    • SVD and PCA
    • Maximum likelihood factor analysis
    • Probabilistic PCA
    • Mixture of factor analyzers
    • Mixture of probabilistic principal component analyzers


    Lecture 6
    Regression

    • Regression
    • Linear models for regression
    • Least suares and RLS
    • Bias-variance dilemma
    • Bayesian linear regression


    • Chapter 3 of Bishop's
    Lecture 7
    Linear models for classification

    • Bayes decision theory
    • Fisher's linear discriminant analysis
    • Logistic regression
    • Perceptron
    • Support vector machine


    Lecture 8
    Neural networks

    • Multilayer perceptron (MLP)
    • Radial basis functoin (RBF) network


    • Chapter 5 of Bishop's

    Lecture 9
    Mixture of experts

    • Mixture of experts (MoE)


    • Chapter 14.5 of Bishop's

    Lecture 10
    Kernel methods

    • Kernel PCA (KPCA)
    • Kernel Fisher discriminant analysis (KFDA)


    • Chapter 12.3 of Bishops'

    Lecture 11
    Hidden Markov models

    • Hidden Markov models (HMMs)


    • Chapter 13.2 of Bishops'

    Lecture 12
    Gaussian process regression
    • xxxxx

    .

  • Homework Assignments