Cross-Validation

Question 1

K-fold cross-validation works by…

Accepted Answer

Rotating k folds for train and validation — Each fold serves as validation once; averaging the k scores gives a more reliable estimate of generalization than a single split.

Question 2

The main advantage of cross-validation over a single train/test split is…

Accepted Answer

A lower-variance, more reliable estimate — Averaging over multiple folds reduces the dependence on one lucky/unlucky split, making the performance estimate more stable.

Question 3

Why keep a separate TEST set even when using cross-validation for tuning?

Accepted Answer

Repeated tuning overfits to the folds — Hyperparameter search optimizes against the validation folds, so a held-out test set is needed for an honest final performance number.

Question 4

STRATIFIED k-fold is important for…

Accepted Answer

Classification tasks with class imbalance — Stratification ensures each fold has a representative class ratio, preventing folds with too few minority-class samples.

Question 5

Leave-One-Out CV (LOOCV) uses k equal to…

Accepted Answer

A k equal to the full sample count n — LOOCV trains n times, each leaving out a single example. It's nearly unbiased but high-variance and computationally expensive.

Question 6

For TIME-SERIES data, standard k-fold is inappropriate because…

Accepted Answer

Shuffling leaks future information — Temporal order matters: you must train on past and validate on future. Time-series CV grows the training window forward without shuffling.

Question 7

A common default for k in k-fold CV is…

Accepted Answer

A k value of 5 or 10 — k=5 or 10 balances bias and variance of the estimate against compute cost, and is the standard choice.

Question 8

NESTED cross-validation is used to…

Accepted Answer

An unbiased estimate while tuning parameters — An inner CV selects hyperparameters and an outer CV estimates generalization, preventing the optimism of tuning and evaluating on the same folds.

Question 9

Larger k in k-fold CV generally gives an estimate with…

Accepted Answer

Lower bias, higher variance, and more compute — More folds means more training data per fold (lower bias) but the estimates are more correlated/variable and costlier (k trainings).

Question 10

A serious mistake that inflates CV scores is performing preprocessing (e.g. scaling, feature selection) …

Accepted Answer

On the full dataset before splitting it — Fitting transforms on all data lets validation information leak into training; preprocessing must be fit within each training fold (use a pipeline).