Machine Learning Fundamentals · 免费预览

反向传播

反向传播沿反向计算图应用链式法则，每次计算一个梯度，并将每个结果复用于所有使用它的环节，因此网络的梯度计算成本与前向传播大致相同。

1. The chain rule, applied in reverse

The chain rule, applied in reverse

2. One reverse pass in detail

One reverse pass in detail

3. Forward and backward as one loop

Forward and backward as one loop

4. Which stored value the output-weight gradient needs

In the reverse pass through the output layer, the gradient $\partial J/\partial \mathbf{W}^{(2)} = (\partial J/\partial \mathbf{o})\,\mathbf{h}^\top$ is computed. The vector $\mathbf{h}$ comes from which phase?

5. The gradient through the softmax and cross-entropy

For a softmax classifier with cross-entropy loss, the gradient of the objective with respect to the output scores $\mathbf{o}$ equals ____.

进入交互式练习

免费注册 · 即时批改 · 自动安排间隔复习。

免费开始这门课