Hallucinations: Causes & Mitigation

Question 1

Why does a language model produce fluent but false claims about a topic it doesn't actually know?

Accepted Answer

Trained for likely tokens, not verified truth — Training optimizes next-token likelihood. When the premise is out of distribution, the top continuations remain fluent and specific — you get confident text without a truth check.

Question 2

You built a customer-support bot on top of RAG. Recall@5 is 0.42. The model still hallucinates. What do you fix first?

Accepted Answer

Fix retrieval quality first — Below ~0.7 recall the generator is largely being asked to answer without evidence. Fixing retrieval is the biggest lever; other defenses ride on top of it.

Question 3

The retrieved chunk says "refunds within 45 days." The bot answers "refunds within 30 days." Which category and which mitigation?

Accepted Answer

Intrinsic; add an entailment check — Intrinsic hallucination is disagreement with provided context. An entailment gate — every claim must be entailed by its cited span — catches this class specifically.

Question 4

A chunk is silent on mobile support. The bot writes "we support Android and iOS." Which category and which mitigation?

Accepted Answer

Extrinsic; require valid citations — Extrinsic hallucinations are unsupported claims. Enforcing citations turns them into schema violations you can drop or route to review.

Question 5

Which of the following most reduces hallucination on numeric/date questions specifically?

Accepted Answer

Grounding via a tool result — Numbers and dates are the classic failure mode for parametric memory. A tool call turns "guess" into "look up."

Question 6

You add an "I don't know" pathway with an instruction and a small reward. It fires 0% of the time. What is the most likely cause?

Accepted Answer

The retriever always returns something — Abstention needs a real trigger (empty retrieval, low verifier confidence) and a hard check that penalizes fabricating on top of thin evidence. Otherwise the model prefers a plausible answer to admitting ignorance.

Question 7

Self-consistency reduces one-off confabulations by:

Accepted Answer

Sampling k paths then majority vote — Random tail tokens vary across paths; consistent facts show up in the majority vote. It requires enough T to produce different paths.

Question 8

A post-hoc verifier gate checks:

Accepted Answer

Whether each claim is entailed by a span — Verifiers are a second-pass NLI-style check between claims and sources. They catch subtle intrinsic hallucination the generator missed.

Question 9

Which metric best tracks extrinsic hallucination in production?

Accepted Answer

Cited-claim coverage rate — Cited-claim rate directly measures whether the model is grounding its assertions. It rises when you tighten citation enforcement and falls when the model starts inventing again.

Question 10

A stakeholder demands "zero hallucinations." Best response?

Accepted Answer

Reframe as a residual-risk budget — Zero is not a shipping goal — error rate is bounded from below by ambiguity. Give the business a measurable budget and a stack of defenses instead of a promise.

Key insight

Cause or cure?

Test your knowledge

Why does a language model produce fluent but false claims about a topic it doesn't actually know?

Interview Questions

References & Further Reading

Related topics

Love this visualization?