Machine Learning Fundamentals · free preview

The Analytic Solution

Linear regression has a closed-form minimizer, which is the exception in machine learning rather than the rule, and it exists only when the design matrix has independent columns.

1. Turning the bias into a weight

Turning the bias into a weight

2. Setting the derivative to zero

Setting the derivative to zero

3. The condition for a unique answer

The condition for a unique answer

4. What breaks the closed form

A design matrix X\mathbf{X} has 200 rows and 5 columns, but two of its columns are exact duplicates of each other. The normal equations X⊤Xw=X⊤y\mathbf{X}^\top\mathbf{X}\mathbf{w} = \mathbf{X}^\top\mathbf{y} are formed. Which statement is correct?

5. The shape of the cross-product

A design matrix X\mathbf{X} has shape (n,d)=(1500,9)(n, d) = (1500, 9). The matrix X⊤X\mathbf{X}^\top\mathbf{X} has shape ____.

Practice this interactively

Free account · instant grading · spaced review that schedules itself.

Start this course for free