Statistical Learning
Modeling, Prediction and Computing
STAT 535 Autumn Quarter 2011


Course Description


Books and other resources

Class mailing list



Handouts/Course notes

UW Statistics


  • Introduction: what is statistical learning (1 lecture)
    • supervised and unsupervised learning
    • a little history
    • the curse of dimensionality
  • Basics: statistical inference and probabilistic independence (2 lectures)
    • multivariate distributions, sample space and random variables
    • operations: inference, sampling, maximum liklihood config, ..
    • examples and applications
  • Graphical Models (4 lectures)
    • graphical representations of conditional independence
    • directed and undirected graphical models
    • Markov properties
    • expression of joint distribution
    • factor graphs
    • log-linear models
    • conditional independence; d-maps and i-maps
    • explaining away; correlation and causation
    • latent vs. observed variables
    • entropy and mutual information as measures of edge strength
  • Inference in graphical models (4 lectures)
    • inference as summation over configurations
    • using graph structure to simplify calculations
    • variable elimination
    • moral, chordal, and decomposable graphs; triangulation
    • the junction-tree algorithm
    • computational complexity, including tree width, cut-sets and phase transitions
  • Approximate inference and sampling (~3 lectures)
    • belief propagation and message passing algorithms
    • forward sampling and importance sampling
    • Gibbs sampling, the Swendsen-Wang algorithm
  • Model estimation (4 lectures)
    • Parameter estimation: mutinomial distribtions, iterative proportional fitting
    • Priors for parameters
    • induction of tree graphs
    • structure estimation
  • Clustering (4 lectures)
    • Model based clustering and the EM algorithm
    • Clustering as optimization
    • Clustering on graphs
    • Cluster validation and finding the number of clusters
  • Guest Lectures -- if time permits