Feature Scaling

Question 1

Why is feature scaling important for KNN?

Accepted Answer

Distance-based, so larger ranges dominate — Since KNN relies on distance calculations (e.g., Euclidean), features with large numerical ranges will disproportionately influence the distance.

Question 2

Standardization (z-score scaling) transforms a feature to have…

Accepted Answer

A mean of 0 and standard deviation 1 — z = (x − μ)/σ centers the feature at 0 with unit variance, putting all features on a comparable scale.

Question 3

Min-max normalization rescales a feature to…

Accepted Answer

A fixed range, typically [0, 1] — x' = (x − min)/(max − min) maps values into [0,1], preserving the shape of the distribution but bounding the range.

Question 4

Which algorithm family is MOST sensitive to feature scaling?

Accepted Answer

Distance-based and gradient-based methods — Methods using distances or gradient steps are dominated by large-range features unless scaled; trees, which split on thresholds, are not.

Question 5

Why does feature scaling help gradient descent?

Accepted Answer

A better-conditioned optimization surface — Unequal scales create elongated loss contours that zig-zag; scaling makes them more spherical, allowing larger, more direct steps.

Question 6

Which models generally do NOT require feature scaling?

Accepted Answer

Tree-based models — Trees split on per-feature thresholds, so monotonic rescaling doesn't change the splits — scaling is unnecessary for them.

Question 7

To avoid DATA LEAKAGE, the scaler should be fit on…

Accepted Answer

Only the training set, then applied elsewhere — Computing μ/σ (or min/max) using test data leaks information; fit on train, transform train and test with those same statistics.

Question 8

Robust scaling is preferred when data has…

Accepted Answer

Outliers — Median and interquartile range are insensitive to extreme values, so robust scaling isn't distorted by outliers the way z-score can be.

Question 9

A log transform is useful for features that are…

Accepted Answer

Highly skewed features — Taking logs compresses large values and spreads small ones, reducing right-skew and making relationships more linear.

Question 10

After standardization, a value of z = 2 means the original was…

Accepted Answer

2 standard deviations above the mean — z-scores are in standard-deviation units, so z=2 is two σ above the mean — a useful, scale-free interpretation.

Gradient Descent: Unscaled vs. Scaled Loss Surface

Scaler Cheatsheet — click to select

Does this model care about scale?

Test your knowledge

Why is feature scaling important for KNN?

Interview Questions

References & Further Reading

Love this visualization?