Class 16: Generalization Error and Stability
Instructor: Lorenzo Rosasco
Description
We review the generalization bounds and use stability to prove generalization bounds for Tikhonov regularization in RKHS and iterative regularization, e.g. Stochastic Gradient Descent, via early stopping.
Class Reference Material
L. Rosasco, T. Poggio, Machine Learning: a Regularization Approach, MIT-9.520 Lectures Notes, Manuscript, Dec. 2017
Chapter 2 - Foundational Results
Note: The course notes, in the form of the circulated book draft is the reference material for this class. Related and older material can be accessed through previous year offerings of the course.
Suggested Reading
- O. Bousquet and A. Elisseeff, Stability and Generalization, Journal of Machine Learning Research, 2002.
- M. Hardt, B. Recht and Y. Singer, Train faster, generalize better: Stability of stochastic gradient descent, International Conference on Machine Learning (ICML), 2016.