Loss Functions

Question 1

A loss function measures…

Accepted Answer

How far the model's predictions are from the true targets — The loss quantifies prediction error; training minimizes it via gradient descent, so it defines what 'good' means for the model.

Question 2

Mean Squared Error (MSE) is the standard loss for…

Accepted Answer

Regression — MSE averages squared residuals (ŷ−y)², penalizing large errors heavily — appropriate for continuous outputs.

Question 3

Cross-entropy loss is used for…

Accepted Answer

Classification — Cross-entropy measures the divergence between the predicted probability distribution and the true (often one-hot) labels.

Question 4

Compared to MSE, Mean Absolute Error (MAE) is…

Accepted Answer

Robust to large outliers because its error penalty grows linearly — MAE grows linearly with error, so large outliers have less disproportionate influence than under MSE's squared penalty.

Question 5

The MAE loss curve is V-shaped rather than parabolic because…

Accepted Answer

Absolute value is piecewise linear, producing constant-slope arms on each side of the target — |y − ŷ| is piecewise linear: constant slope −1 to the left of the target, constant slope +1 to the right. This contrasts with MSE's parabola, where gradient magnitude grows proportionally to error size.

Question 6

Binary cross-entropy is paired with which output activation?

Accepted Answer

Sigmoid — Sigmoid outputs P(class=1); binary cross-entropy is its matching negative-log-likelihood loss.

Question 7

Categorical cross-entropy is paired with…

Accepted Answer

Softmax — Softmax + categorical cross-entropy is the standard multiclass setup, yielding the clean (p − y) gradient.

Question 8

Why does cross-entropy heavily penalize confident wrong predictions?

Accepted Answer

As predicted probability for the correct class nears zero, −log(p) grows toward infinity — Assigning near-zero probability to the correct class incurs a huge log loss, strongly discouraging overconfident mistakes.

Question 9

Hinge loss is the loss function associated with…

Accepted Answer

Support Vector Machine classification — Hinge loss penalizes points within or on the wrong side of the margin, producing the SVM's maximum-margin solution.

Question 10

A loss function used for training must generally be…

Accepted Answer

Differentiable with respect to model predictions — Gradient-based optimization needs derivatives; non-differentiable metrics (like accuracy) are evaluated but not directly minimized.

Loss Functions Visualizer

Mean Squared Error Curve

Function Details

Which loss for which task?

Test your knowledge

A loss function measures…

Interview Questions

References & Further Reading

Love this visualization?