Machine Learning Fundamentals · 免费预览

病态损失函数下的动量

当损失曲线在一个方向上比另一个方向上陡峭得多时，梯度下降无法采取在每个方向上都安全的步长，而动量通过对过去的梯度取平均，使其继续沿平坦的方向前进。

1. A loss with a ravine

A loss with a ravine

2. Averaging the past gradients

Averaging the past gradients

3. What momentum does and does not fix

What momentum does and does not fix

4. Why momentum helps on a ravine

Gradient descent on f(x)=0.1x12+2x22f(\mathbf{x}) = 0.1x_1^2 + 2x_2^2 zigzags, bouncing across the steep direction while advancing slowly along the flat one. Which statement describes what momentum changes?

5. The velocity update

With a momentum coefficient β=0.9\beta = 0.9, a previous velocity of 00 and a current gradient of 22, the new velocity vt=βvt−1+∇Lv_t = \beta v_{t-1} + \nabla \mathcal{L} equals ____.

进入交互式练习

免费注册 · 即时批改 · 自动安排间隔复习。

免费开始这门课