Machine Learning Fundamentals · free preview
Distribution Shift
A classifier inherits the assumptions of the data it was fitted on, and when the test distribution moves in any of three specific ways, accuracy degrades in a way that a held-out set cannot detect.
1. The setup
The setup
2. Covariate shift
Covariate shift
3. Label shift and concept shift
Label shift and concept shift
4. Which distribution shifted
A spam classifier is trained on email from one country where 10% of messages are spam, and deployed in another country where 60% are spam. The visual characteristics of a spam message are the same in both countries. Which type of shift is this?
5. Covariate shift holds which factor fixed
Under covariate shift, the distribution of inputs may change between training and test, while the labeling function ____ stays fixed.
Practice this interactively
Free account · instant grading · spaced review that schedules itself.
Start this course for free