Reasoning Models & Test-Time Compute

Question 1

What is the "thinking" or "reasoning trace" a model emits?

Accepted Answer

Hidden chain-of-thought the model emits before its answer — The model drafts intermediate reasoning — exploring, checking, backtracking — before committing to an answer. Most APIs hide or summarize it.

Question 2

Test-time compute improves answers by ___.

Accepted Answer

spending more inference tokens per question at decode time — It scales a second axis: hold the model fixed and buy accuracy with more inference-time reasoning, instead of training a bigger model.

Question 3

How are thinking tokens typically billed?

Accepted Answer

At the output-token rate — Thinking is generated text — output tokens, the expensive kind — even though the user never reads most of it.

Question 4

The accuracy-vs-thinking-budget curve typically looks like:

Accepted Answer

Steep early gains that level off into a plateau — The first few thousand thinking tokens buy most of the gain; past the knee the bill keeps rising while accuracy barely moves.

Question 5

Which task is the BEST fit for a reasoning model?

Accepted Answer

Debugging an intermittent race condition in shared state — Debugging needs hypothesis elimination across steps where errors compound — exactly what deliberation buys. The others are single-hop recognition or mechanical transforms.

Question 6

Which task is the WORST fit for a reasoning budget?

Accepted Answer

Latency-sensitive customer support chat — Chat users notice a multi-second pause more than a marginally better sentence — thinking adds latency and cost without moving the outcome.

Question 7

A production router in front of reasoning models typically:

Accepted Answer

Defaults to a standard model and escalates hard or failed cases — Tiering is the cost pattern: cheap model for the many easy calls, thinking reserved for the hard slice or for retries after a failed check.

Question 8

Why does a large thinking budget hurt user experience even when accuracy improves?

Accepted Answer

Thinking runs first, so users wait seconds before seeing any answer token — Thinking streams (or happens) first — a 20k-token think is ~10-15 seconds before the user sees any of the actual answer.

Question 9

How do reasoning models differ from ordinary chain-of-thought prompting?

Accepted Answer

Reasoning models are trained (often with RL) to reason, not just prompted to — "Think step by step" elicits reasoning from a standard model; reasoning models are explicitly trained to produce long, self-correcting deliberation and to use a token budget well.

Question 10

What does the "effort" or "budget" parameter on reasoning APIs control?

Accepted Answer

How many thinking tokens the model may spend per call — It caps deliberation — low/medium/high effort or an explicit max thinking-token count — and is your main cost/latency dial per route.

Key insight

Where does the extra compute go?

Test your knowledge

What is the "thinking" or "reasoning trace" a model emits?

The short version

In more depth

Reasoning Models & Test-Time Compute

How it works

Engineering rules of thumb

Interview Questions

References & Further Reading

Related topics

Love this visualization?