Gradient Descent and Minibatches

The algorithm that trains everything

The gradient of the linear model

Full batch, single example, or something between

Hyperparameters

In linear regression, the gradient contribution of example $i$ is $\mathbf{x}^{(i)}(\mathbf{w}^\top\mathbf{x}^{(i)} + b - y^{(i)})$. For an example whose prediction exactly equals its label, what happens on the next update?

The update rule is $\mathbf{w} \leftarrow \mathbf{w} - \eta \nabla_{\mathbf{w}}\mathcal{L}$. With $\eta = 0.05$ and a gradient component of $+4$ along one coordinate, that coordinate changes by ____ on the step.