 Handout 0  About the course

Lecture 1  Statistical prediction  an overview (11 pages)

Lecture 2  Optimization in statistics (9 pages)
 Lecture 3part I  Unconstrained optimization (part I, 11 pages)
 Lecture 3part II  Unconstrained optimization (part II, 17 pages)
 The perceptron algorithm lecture notes by Avrim Blum
 Lecture 4  Boosting and other methods for averaging classifiers (26 pages) 2/7/12 Updated with extensions to Boosting pages 2326. typos fixed on pages 1314
 A Boosting Approach to Machine Learning by Rob Schapire (an introductory article)

"AdaBoost is consistent" by Peter Bartlett and Mikhail Traskin
 ''Convexity, classification, and risk bounds'' by Peter
L. Bartlett, Michael I. Jordan, Jon D. McAuliffe, in Journal of the
American Statistical Association 101(473), March 2006: 138156.
 "Evidence Contrary to the Statistical View of Boosting" by David Mease, Abraham Wyner, JMLR 2008
(a set of experiments highlighting unanswered questions about
boosting)
 Large margins using the perceptron algorithm" by Yoav Freund and Rob Schapire, (The Voted Perceptron Algorithm) in Machine Learning 37(3):277296, 1999
 Maryam Fazel's slides on convex optimization. Look at the Convex sets and Convex functions lectures.
 Lecture 5 Convex sets (a stump. Small additions to Boyd Chapter 2)
 Lecture 6 Convex functions (a very small stump. Small additions to Boyd Chapter 3)
 Lecture 7 Convex conjugate, Bregman divergences, exponetial family models (7 pages)
 Generalized linear models: A unified approach by Jeff Gill
"Clustering with Bregman divergences" by Banerjee et al. (45 pages; of main interest sections 2, 3 and appendix A)
 Lecture 8 Maximum entropy (20 pages)
 Lecture 9 Support Vector Machines (16 pages)

"A tutorial on support vector machines for pattern recognition" by Chris Bur
gess (appeared in Data Mining and Knowledge Discovery, 2, 121167, 1998.
 www.kernelmachines.org The SVM and Kerne
l machines resource page (not up to date)
 E. J. Candes and M. Wakin. An introduction to compressive sampling IEEE Signal Processing Magazine, March 2008 2130.
