Decision Tree

Question 1

A decision tree makes predictions by…

Accepted Answer

Recursively splitting on chosen thresholds — Each internal node tests a feature; you follow branches based on the answer until reaching a leaf, which holds the prediction.

Question 2

Which criterion is commonly used to choose splits in classification trees?

Accepted Answer

Gini impurity or information gain — Splits are chosen to maximally reduce impurity — Gini or entropy/information gain — making child nodes purer than the parent.

Question 3

Decision tree training is…

Accepted Answer

Greedy — Finding the globally optimal tree is NP-hard, so trees grow greedily, choosing the best immediate split (which may not be globally optimal).

Question 4

Why do unpruned decision trees tend to OVERFIT?

Accepted Answer

They grow until each leaf is pure — A fully grown tree can isolate every training point; pruning, max-depth, or min-samples-per-leaf limits this to improve generalization.

Question 5

Decision trees produce decision boundaries that are…

Accepted Answer

Axis-aligned rectangular regions — Each split is a threshold on one feature, so the boundary is a staircase of axis-parallel cuts — flexible but blocky.

Question 6

A key advantage of decision trees is that they are…

Accepted Answer

Interpretable — The rule path is human-readable, making trees a 'white-box' model valued where explanations matter.

Question 7

Do decision trees require feature scaling/normalization?

Accepted Answer

No, thresholds are scale-invariant — Since splits use thresholds on individual features, scaling features doesn't affect which splits are chosen — a practical convenience.

Question 8

For REGRESSION trees, leaves predict…

Accepted Answer

The mean — Regression trees split to reduce variance and output the average target value per leaf, giving a piecewise-constant function.

Question 9

A major weakness of single decision trees is that they are…

Accepted Answer

High-variance/unstable — Instability is why trees are combined into ensembles (random forests, boosting) that average away this variance.

Question 10

Information gain from a split is defined as…

Accepted Answer

Parent entropy minus weighted child entropy — Higher information gain means the split makes the children purer; the algorithm picks the feature/threshold maximizing it.

Decision Tree Visualizer
Depth: 2

Tree Parameters

Controls

Tree Info

Decision Space

Tree Structure

Gini Impurity

Entropy

Information Gain

Grow one node

Test your knowledge

A decision tree makes predictions by…

Interview Questions

References & Further Reading

Love this visualization?

Decision Tree

Decision Tree VisualizerDepth: 2

Tree Parameters

Controls

Tree Info

Decision Space

Tree Structure

Gini Impurity

Entropy

Information Gain

Grow one node

Test your knowledge

A decision tree makes predictions by…

Interview Questions

References & Further Reading

Love this visualization?

Decision Tree Visualizer
Depth: 2