Overfitting & Regularization

Question 1

Overfitting is when a model…

Accepted Answer

Memorizes training noise and fails to generalize — An overfit model memorizes idiosyncrasies/noise of the training set, so test performance lags far behind training performance.

Question 2

The hallmark symptom of overfitting is…

Accepted Answer

Low training error but high validation/test error — A growing gap between training and validation error signals the model is fitting noise rather than generalizable structure.

Question 3

Regularization techniques primarily reduce…

Accepted Answer

Variance — By penalizing complexity or adding noise, regularization makes the model less sensitive to the particular training set.

Question 4

L2 regularization (weight decay) adds a penalty proportional to…

Accepted Answer

The sum of squared weights — L2 shrinks weights smoothly toward (but not exactly to) zero, reducing variance and improving generalization.

Question 5

L1 regularization is notable for…

Accepted Answer

Driving some weights exactly to zero, enabling feature selection — The L1 penalty's geometry pushes coefficients to exactly zero, yielding sparse, interpretable models.

Question 6

DROPOUT regularizes a neural network by…

Accepted Answer

Randomly deactivating a fraction of neurons each training step — Dropping random units prevents co-adaptation and forces redundancy, acting like training an ensemble of subnetworks.

Question 7

EARLY STOPPING prevents overfitting by…

Accepted Answer

Halting training when validation performance stops improving — It stops before the model starts fitting noise, using the validation curve to pick the best-generalizing checkpoint.

Question 8

Data augmentation reduces overfitting by…

Accepted Answer

Generating transformed copies of training examples to add diversity — Augmentation exposes the model to more variation, improving invariance and generalization — especially in vision/audio.

Question 9

Simply collecting MORE training data helps overfitting by…

Accepted Answer

Reducing memorization since no single example can dominate — More data better represents the true distribution, so the model must learn general patterns rather than memorizing examples.

Question 10

Underfitting (the opposite problem) is fixed by…

Accepted Answer

Increasing model capacity, reducing regularization, or adding features — High bias needs a more expressive model or less constraint — the reverse of the overfitting remedies.

Overfitting & Regularization

Regularization Method

Strength Parameter

L2 Regularization (Ridge)

Current Metrics

Without Regularization

L2 Regularization

Bias-Variance Tradeoff

How L2 Regularization (Ridge) Works

Overfitting Signs

When to Regularize

λ Tuning Tips

L1 or L2?

Test your knowledge

Overfitting is when a model…

Learn these first

Interview Questions

References & Further Reading

Love this visualization?