Class 03: Regularized Least Squares

Instructor: Lorenzo Rosasco

Description

We go from Ordinary Least Squares to Ridge Regression, viewed in two different ways, and discuss regularization and bias.

Slides

Slides for this lecture: PDF

Class Reference Material

L. Rosasco, T. Poggio, Machine Learning: a Regularization Approach, MIT-9.520 Lectures Notes, Manuscript, Dec. 2017

Chapter 4 - Regularization Networks

Note: The course notes, in the form of the circulated book draft is the reference material for this class. Related and older material can be accessed through previous year offerings of the course.

Further Reading

• R. Rifkin and R. A. Lippert, Notes on Regularized Least-Squares, CBCL Paper #268/AI Technical Report #2007-019, MIT, May, 2007.
• E. D. Vito, L. Rosasco, A. Caponnetto, U.D. Giovannini and F. Odone, Learning from examples as an inverse problem, Journal of Machine Learning Research 6 (May), 883-904, 2005.
• S. Mosci, L. Rosasco, and A. Verri, Dimensionality reduction and generalization, Proc. of the 24th international conference on Machine learning, 2007.