Machine Learning Fundamentals · 免费预览

梯度下降与小批量

大多数模型都是通过迭代地沿损失函数梯度的反方向更新参数来训练的，而用于估计该梯度的批大小则是在统计精度与速度之间进行权衡。

1. The algorithm that trains everything

The algorithm that trains everything

2. The gradient of the linear model

The gradient of the linear model

3. Full batch, single example, or something between

Full batch, single example, or something between

4. Hyperparameters

Hyperparameters

5. What the residual does to the gradient

In linear regression, the gradient contribution of example ii is x(i)(w⊤x(i)+b−y(i))\mathbf{x}^{(i)}(\mathbf{w}^\top\mathbf{x}^{(i)} + b - y^{(i)}). For an example whose prediction exactly equals its label, what happens on the next update?

6. The learning rate direction

The update rule is w←w−η∇wL\mathbf{w} \leftarrow \mathbf{w} - \eta \nabla_{\mathbf{w}}\mathcal{L}. With η=0.05\eta = 0.05 and a gradient component of +4+4 along one coordinate, that coordinate changes by ____ on the step.

进入交互式练习

免费注册 · 即时批改 · 自动安排间隔复习。

免费开始这门课