Bias-Variance Tradeoff

Question 1

What is high bias usually associated with?

Accepted Answer

Underfitting — High bias means the model is too simple to capture the underlying pattern (underfitting).

Question 2

What is high variance usually associated with?

Accepted Answer

Overfitting — High variance means the model captures noise in the training data and generalizes poorly (overfitting).

Question 3

In the bias-variance decomposition, BIAS refers to error from…

Accepted Answer

Overly simplistic assumptions — High bias means the model is too simple to capture the underlying relationship, systematically erring — underfitting.

Question 4

VARIANCE refers to error from…

Accepted Answer

Sensitivity to the training set — High variance means the model changes a lot with different training data — it fits noise, generalizing poorly (overfitting).

Question 5

Expected prediction error decomposes into…

Accepted Answer

Bias squared, variance, and irreducible error — Total error = bias² + variance + irreducible (noise). You can reduce bias and variance but never the irreducible noise floor.

Question 6

As model COMPLEXITY increases, typically…

Accepted Answer

Bias falls while variance rises — More complex models fit training data better (lower bias) but become more sensitive to it (higher variance) — the core trade-off.

Question 7

A model with HIGH BIAS shows…

Accepted Answer

High error on both training and test sets — Underfitting models can't even fit the training data well, so both training and validation errors are high and close together.

Question 8

A model with HIGH VARIANCE shows…

Accepted Answer

Low training error but high test error — Overfitting yields near-zero training error but poor test error; the gap between them signals high variance.

Question 9

The 'irreducible error' comes from…

Accepted Answer

Inherent noise no model removes — Measurement noise and inherent randomness set a floor on achievable error regardless of the model.

Question 10

Adding MORE TRAINING DATA primarily reduces…

Accepted Answer

Variance — More data stabilizes estimates, lowering variance/overfitting. It does little for bias if the model itself is too simple.