Classic methods of Machine Learning
STAT 592 / CSE 590 MM Winter Quarter 2004



Books, software and other resources

Class mailing list



Course notes and reading

UW Statistics

UW Computer Science


Below is a tentative syllabus. The numbers in brackets [] correspond to "optional" topics. We will choose which of them to cover based on the interests of the class participants.

I. Supervised learning (Classification)

1. classification with generative models

2. classification with discriminative models:
-nearest neighbor classifiers
-classification (and regression) trees
-Support Vector Machines

II. Unsupervised learning

3. Clustering: model based clustering, the EM algorithm
-recent advances: the 2-iterations EM algorithm,
random projections based algorithms, infinite mixtures

[4.] Discrete multi-dimensional probabilities: the Boltzmann machine

5. Dimension reduction: Principal component analysis, random projection, non-linear dimension reduction (kernel PCA, locally linear embedding)

III. Advanced and Information theoretic techniques

6. Boosting

7. Entropy and mutual information and their relation to statistics.
Maximum entropy models

"Between the lines" we will discuss also:

  • Entropy and information
  • Bayesian learning
  • Model selection methods: Minimum Description Length/MML/Akaike, Structural risk minimization and VC bounds, Cross-Validation, etc
  • Convex optimization
  • Non-parametric vs parametric methods