## Module I: Preliminaries of Nonparametric Regression
__[-]__
__Collapse All[-]__

- Introduction: course overview; example tasks
- Optimal Predictions and Measures of Accuracy: loss functions; predictive risk; bias-variance trade-off
- Linear Smoothers: definition; basic examples
- A First Look at Shrinkage Methods: ridge regression; lasso
- Choosing the Smoothing Parameter: analytic approaches; cross validation

### Lectures:

- 1. Apr 1: Intro. Optimal predictions, predictive performance, bias-variance tradeoff.

[Intro, predictions, bias-variance tradeoff slides] [Intro, predictions, bias-variance tradeoff annotated slides] - 2. Apr 3: Linear smoothers, ridge regression, LASSO.

[Linear smoothers, ridge regression, LASSO slides] [Linear smoothers, ridge regression, LASSO annotated slides] - 3. Apr 7: LASSO cont'd.

[LASSO cont'd slides] [LASSO cont'd annotated slides]

## Module II: Splines and Kernel Methods
__[-]__
__Collapse All[-]__

- Introduction: brief overview
- Spline Methods: piecewise polynomials; natural cubic splines; smoothing splines; B-splines; penalized regression splines
- Kernel Methods: kernel density estimation; the Nadaraya-Watson kernel estimator; local polynomial regression
- Inference for Linear Smoothers: variance estimation; confidence bands
- Spline and Kernel Methods for GLMs: extensions of spline and kernel methods to binomial, Poisson, gamma, etc, data

### Lectures:

- 4. Apr 8: Smoothing parameters, spline intro

[Smoothing parameters, spline slides][Smoothing parameters, spline annotated slides] - 5. Apr 10: Natural splines, smoothing splines, B-splines, penalized regression splines

Note: Lecture cut short by fire alarm.

[Natural, smoothing, penalized regression, B- splines slides][Natural, smoothing, B- splines annotated slides] - 6. Apr 15: Penalized regression splines (carry-over from fire alarm). Kernel methods intro, local polynomial regression

[Penalized regression, kernels, local polynomial regression slides][Penalized regression, kernels, local polynomial regression annotated slides] - 7. Apr 17: KDE, inference in linear smoothers.

[KDE, inference in linear smoothers slides][KDE, inference in linear smoothers annotated slides]

## Module III: Bayesian Nonparametrics
__[-]__
__Collapse All[-]__

- Introduction: principles of Bayesian nonparametrics
- Regression via Gaussian processes
- Density estimation via Dirichlet process mixture of Gaussians

### Lectures:

- 8. Apr 21: Intro to Gaussian processes

[Intro to Gaussian processes slides] [Intro to Gaussian processes annotated slides] - 9. Apr 22: Gaussian processes cont'd, GPs for regression

[Gaussian processes, GPs for regression slides][Gaussian processes, GPs for regression annotated slides] - 10. Apr 24: Selecting GP hyperparameters, GP recap, finite mixture models

[GP hypers+recap, mixture models slides] [GP hypers+recap, mixture models annotated slides] - 11. Apr 29: Gibbs sampling, Dirichlet process mixture models

[Gibbs sampling, Dirichlet process mixture models slides][Gibbs sampling, Dirichlet process mixture models annotated slides]

## Module IV: Nonparametrics with Multiple Predictors
__[-]__
__Collapse All[-]__

- Introduction: issues when considering multiple predictors
- Generalized Additive Models: GAMs; the backfitting algorithm
- Spline Methods in Several Variables: natural thin plate splines; thin plate regression splines; tensor product splines
- Kernel Methods in Several Variables: extending kernel methods to multidimensional covariates
- Smoothing Parameter Estimation: how to choose level of smoothing in more than one dimension
- Regression Trees: partitioning the covariate space

### Lectures:

- 12. May 1: Multidimensional splines

[Multidimensional splines slides] [Multidimensional splines annotated slides] - 13. May 5: Multidimensional kernel methods, projection pursuit

[Multidimensional kernel method and projection pursuit slides] [Multidimensional kernel method and projection pursuit annotated slides] - 14. May 6: Regression trees

[Regression tree slides] [Regression tree annotated slides]

## Module V: Classification
__[-]__
__Collapse All[-]__

- Logistic Regression
- Bayes Classifiers: linear and quadratic classifiers; naive Bayes classifiers using KDE
- Perceptrons for online learning and SVMs
- Boosting

### Lectures:

- 15. May 8: MARS, classification trees

[MARS, classification trees slides] [MARS, classification trees annotated slides] - 16. May 13: Classification intro, logistic regression

[Classification intro, logistic regression slides] [Classification intro, logistic regression annotated slides] - 17. May 15: LDA, QDA, KDE for classification, and Naive Bayes

[LDA, QDA, KDE, and Naive Bayes slides] [LDA, QDA, KDE, and Naive Bayes annotated slides] - 18. May 19: Mixture models, online learning, and perceptron algorithm

[Online learning and perceptron slides][Online learning and perceptron annotated slides] - 19. May 20: Kernelized perceptron and SVMs

[Kernelized perceptron and SVM slides] [Kernelized perceptron and SVM annotated slides] - 20. May 22: Multiclass SVMs and boosting