Machine Learning Fundamentals · 免费预览

交叉熵及其梯度

Softmax 交叉熵是正确类别的负对数似然，其关于分数的梯度与线性回归的梯度形状相同：特征向量乘以一个残差。

1. From probability to loss

From probability to loss

2. The simplified form

The simplified form

3. The gradient

The gradient

4. What the loss charges

A three-class model assigns probability y^2=0.7\hat{y}_2 = 0.7 to the true class, whose one-hot label is (0,1,0)(0, 1, 0). The per-example loss is −∑jyjlog⁡y^j-\sum_j y_j \log \hat{y}_j. The value of that loss is closest to which number?

5. The gradient at the true class

A model's softmax output for the true class is y^2=0.7\hat{y}_2 = 0.7 with label y2=1y_2 = 1. The gradient component ∂l/∂o2=y^2−y2\partial l / \partial o_2 = \hat{y}_2 - y_2 equals ____.

进入交互式练习

免费注册 · 即时批改 · 自动安排间隔复习。

免费开始这门课