Linear Regression

Question 1

What method is commonly used to find the best fit line?

Accepted Answer

Minimizing the residual sum of squares — Linear regression typically minimizes the sum of squared differences (residuals) between observed and predicted values.

Question 2

What are the two parameters learned in simple linear regression?

Accepted Answer

Slope and Intercept — Simple linear regression learns the slope (m) and y-intercept (b) of the line y = mx + b.

Question 3

Linear regression models the relationship between inputs and a continuous output as…

Accepted Answer

A weighted sum plus a bias — It fits a line/hyperplane ŷ = w·x + b to predict a continuous target from features.

Question 4

The loss function typically minimized in linear regression is…

Accepted Answer

Just the mean squared error — MSE penalizes squared residuals, yielding a smooth convex objective whose minimum gives the best-fit line.

Question 5

The MSE loss surface for linear regression is…

Accepted Answer

Convex — MSE is convex in the weights, so there is a single global minimum (no bad local minima), which the normal equation or gradient descent reaches.

Question 6

The closed-form 'normal equation' solution is…

Accepted Answer

w = (XᵀX) inverse, times Xᵀy — Setting the MSE gradient to zero gives w = (XᵀX)⁻¹Xᵀy — exact, but O(d³) to invert, so gradient descent is preferred for large d.

Question 7

A 'residual' in linear regression is…

Accepted Answer

The actual value minus the predicted value — Residuals are the errors the model makes; least squares minimizes the sum of their squares.

Question 8

R² (coefficient of determination) measures…

Accepted Answer

The proportion of variance the model explains — R² ∈ (−∞, 1] compares the model's errors to a baseline (predicting the mean); 1 means a perfect fit, 0 means no better than the mean.

Question 9

Linear regression assumes the relationship between features and target is…

Accepted Answer

Approximately a linear relationship — It models linear effects; non-linear patterns require feature engineering (polynomials, interactions) or non-linear models.

Question 10

Ridge regression adds which penalty to the loss?

Accepted Answer

L2 (Σw²) — Ridge (L2) regularization discourages large weights, reducing variance/overfitting and stabilizing the solution when features are collinear.

Linear Regression

Fit a line

Test your knowledge

What method is commonly used to find the best fit line?

Interview Questions

References & Further Reading

Love this visualization?