I. Supervised learning (Classification)

1. classification with generative models

2. classification with discriminative models:
-nearest neighbor classifiers
-classification (and regression) trees
-Support Vector Machines

II. Unsupervised learning

3. Clustering: model based clustering, the EM algorithm
-recent advances: the 2-iterations EM algorithm,
random projections based algorithms, infinite mixtures

[4.] Discrete multi-dimensional probabilities: the Boltzmann machine

5. Dimension reduction: Principal component analysis, random projection, non-linear dimension reduction (kernel PCA, locally linear embedding)

III. Advanced and Information theoretic techniques

6. Boosting

7. Entropy and mutual information and their relation to statistics.
Maximum entropy models

"Between the lines" we will discuss also:

• Entropy and information
• Bayesian learning
• Model selection methods: Minimum Description Length/MML/Akaike, Structural risk minimization and VC bounds, Cross-Validation, etc
• Convex optimization
• Non-parametric vs parametric methods