Machine Learning Fundamentals · free preview

Why the Squared Loss

Minimizing mean squared error is the same as maximizing the likelihood of the data when noise is Gaussian, which is why one loss serves two justifications.

1. The noise assumption, stated precisely

The noise assumption, stated precisely

2. From likelihood to a sum

From likelihood to a sum

3. What survives

What survives

4. Why the first term drops out

The negative log-likelihood under Gaussian noise contains the term 12log⁡(2πσ2)\frac{1}{2}\log(2\pi\sigma^2) summed over examples, along with the squared-error term. The first term is dropped when minimizing over w\mathbf{w} and bb. Which reason is correct?

5. The likelihood of a single observation

Under Gaussian noise with mean w⊤x+b\mathbf{w}^\top\mathbf{x} + b and variance σ2\sigma^2, the density is proportional to exp⁡ ⁣(−12σ2(y−w⊤x−b)2)\exp\!\left(-\frac{1}{2\sigma^2}(y - \mathbf{w}^\top\mathbf{x} - b)^2\right). With σ=2\sigma = 2, the exponent for an observation with residual y−w⊤x−b=4y - \mathbf{w}^\top\mathbf{x} - b = 4 equals ____.

Practice this interactively

Free account · instant grading · spaced review that schedules itself.

Start this course for free