Machine Learning Fundamentals · free preview

One-Hot Labels and the Softmax

Classification predicts a category, category names carry no order, and the softmax turns a vector of scores into a probability distribution over categories.

1. Encoding a category

Encoding a category

2. One affine function per class

One affine function per class

3. Turning scores into probabilities

Turning scores into probabilities

4. Why raw scores cannot be probabilities

A three-class model produces outputs $\mathbf{o} = (2.0, -1.5, 4.0)$ from its affine layer. These values are passed to a cross-entropy loss without a softmax. Which statement describes the difficulty?

5. Softmax on a two-class case

A binary classifier produces scores $o_1 = 0$ and $o_2 = 0$ . The softmax assigns probability $\hat{y}_1 = \exp(o_1)/(\exp(o_1) + \exp(o_2))$ to class 1. The value of $\hat{y}_1$ is ____.

Practice this interactively

Free account · instant grading · spaced review that schedules itself.

Start this course for free