Naive Bayes Classifier

Question 1

Naive Bayes classifiers are based on which theorem?

Accepted Answer

Bayes' theorem — It applies Bayes' theorem P(y|x) ∝ P(x|y)P(y) to compute the posterior probability of each class given the features.

Question 2

What is the 'naive' assumption in Naive Bayes?

Accepted Answer

Features are conditionally independent — It assumes each feature contributes independently to the class probability — usually false, but the model often works well anyway.

Question 3

Naive Bayes predicts the class with the highest…

Accepted Answer

The posterior probability — It computes P(class|features) ∝ P(class)·∏P(featureᵢ|class) for each class and picks the maximum (the MAP estimate).

Question 4

Why are probabilities multiplied as a sum of LOGS in practice?

Accepted Answer

To avoid numerical underflow — Products of many probabilities (<1) underflow to zero; summing log-probabilities is numerically stable and monotonic, preserving the argmax.

Question 5

What problem does Laplace (additive) smoothing solve?

Accepted Answer

An unseen value zeroing out the product — Adding a small count (e.g. +1) to every feature/class tally ensures no probability is exactly zero, so an unseen word doesn't zero out the posterior.

Question 6

Multinomial Naive Bayes is especially suited to…

Accepted Answer

Text with word counts — It models feature counts (like word frequencies), making it a classic, fast baseline for document/spam classification.

Question 7

Gaussian Naive Bayes assumes each continuous feature follows a…

Accepted Answer

Normal — For real-valued features it estimates a per-class mean and variance and uses the Gaussian density for P(featureᵢ|class).

Question 8

The 'prior' P(class) in Naive Bayes is typically estimated as…

Accepted Answer

The fraction of examples in that class — The prior is the class's empirical frequency; it biases predictions toward more common classes before evidence is considered.

Question 9

What is the training time complexity of Naive Bayes?

Accepted Answer

O(n·d) — It just tallies counts/means per feature per class in one pass over n examples with d features, making it extremely fast to train.

Question 10

Despite its unrealistic independence assumption, Naive Bayes often performs well because…

Accepted Answer

Correct ranking only needs the argmax to be right — Even with biased probability estimates, the class with the highest posterior is frequently still correct, so classification accuracy holds up.

Test Point

Dataset

View Options

Class 0 Parameters

Class 1 Parameters

Live Computation for Test Point (50, 50)

Classify a document with Naive Bayes

Test your knowledge

Naive Bayes classifiers are based on which theorem?

Learn these first

Interview Questions

References & Further Reading

Love this visualization?