Machine Learning Fundamentals · free preview
What Training Optimizes, and What It Cannot
The training loss is an average over a finite sample, and its minimum sits at a different parameter value from the minimum of the expected loss, so optimization alone cannot deliver generalization.
1. Empirical risk and risk
Empirical risk and risk
2. Why stopping early is principled
Why stopping early is principled
3. Two kinds of stationary point
Two kinds of stationary point
4. Why the training loss can mislead
A model's training loss is driven to nearly zero by increasing its capacity, and its validation loss rises once the training loss is below a threshold. Which statement explains the divergence between the two losses?
5. The saddle point test
At a stationary point of a two-dimensional loss, the Hessian has eigenvalues and . The point is a ____.
Practice this interactively
Free account · instant grading · spaced review that schedules itself.
Start this course for free