How Reasoning Models Work

Models trained to think at length before answering. The same token-equals-computation principle, moved from your prompt into the training.

Chain-of-thought prompting works because each token gets one forward pass, so written-out steps are genuine additional computation.

Reasoning models take that principle and train it in. Rather than needing to ask for step-by-step work, the model generates extended reasoning before answering, by default and at much greater length.

What changed

The mechanism is unchanged: more tokens, more computation. Three things are different.

Length. Prompted chain-of-thought produces a few hundred reasoning tokens. Reasoning models produce thousands, sometimes tens of thousands.

Behaviour within the reasoning. They backtrack, check work, notice errors, and try alternative approaches — patterns rarely seen in prompted reasoning.

Training signal. This is the substantive change. Instead of learning to imitate human-written reasoning, they are trained with reinforcement learning on outcomes: generate reasoning, produce an answer, check whether it was correct, reinforce reasoning that led to correct answers.

Nobody demonstrates good reasoning. The model discovers what works. Behaviours like self-correction emerge because they raise the success rate, not because they were taught.

This requires verifiable answers, which is why the strongest gains are in mathematics, competitive programming, and formal logic — domains where correctness can be checked automatically. Progress on tasks without automatic verification has been more modest.

Inference-time scaling

The important consequence: you can trade compute for accuracy at request time.

Historically, better output meant a bigger model — a training-time decision. Reasoning models add a second dial. The same model, allowed to think longer, performs better on hard problems.

Most providers expose this as an effort or budget setting. Low for simple tasks, high for hard ones. This is a genuinely new axis in how these systems are used, and it is the main practical thing to internalize.

The reasoning is usually hidden

Providers typically do not return raw reasoning tokens. You get a summary, or nothing, plus a count of tokens consumed.

Two reasons given: raw reasoning is unpolished and can be misleading, and it represents competitive investment.

You are billed for those tokens regardless, which is the cost model to understand — a reasoning response can consume many times the visible output in tokens.

What they are good at

Multi-step mathematics, where the training signal was strongest.

Competitive programming, especially problems needing algorithmic insight.

Hard debugging — tracing through logic and considering alternatives.

Constraint satisfaction — scheduling, puzzles, anything where constraints interact.

Careful analysis where the failure mode is jumping to conclusions.

What they are not good at

Simple tasks. Extraction, classification, formatting. Reasoning adds cost and latency for no gain, and can produce worse results by overthinking an unambiguous problem.

Latency-sensitive work. Thousands of reasoning tokens take real time. Not suitable for interactive typeahead.

Knowledge gaps. Reasoning cannot recover missing information. It produces elaborate, well-structured wrong answers — arguably worse than a short wrong answer, because the structure is persuasive. RAG is still the fix.

Tasks where the process must be auditable, given hidden reasoning.

The faithfulness question stands

The caveat from chain-of-thought applies here too, and matters more because the reasoning is longer and less visible.

Stated reasoning does not necessarily reflect what determined the answer. Research has shown models reaching conclusions and then producing justifying reasoning, and cases where a hint changes the answer while the reasoning never mentions it.

Extra tokens genuinely improve accuracy. The reasoning trace is not a reliable audit trail. Both are true.

Practical use

Route by difficulty. Reasoning models for hard problems, standard models for the rest. Paying reasoning prices for extraction is the same avoidable waste as using a frontier model for classification.

Do not add “think step by step.” It is trained in. Prompting for it can interfere.

Prompt differently. These models respond better to a clear problem statement than to detailed process instructions. Say what you want; do not choreograph how to think.

Watch the cost. Reasoning tokens are output tokens — the expensive kind — and they are invisible in the response, so budget surprises are common. Log token counts.

What to remember

• Same principle as chain-of-thought — tokens are computation — trained in rather than prompted.
• Trained by reinforcement learning on verifiable outcomes, so self-correction emerged rather than being demonstrated.
• Introduces inference-time scaling: trade compute for accuracy per request.
• Strong on math, competitive programming, debugging, and constraint problems; wasteful on simple tasks.
• Reasoning is usually hidden but always billed; the trace is not a trustworthy audit trail.

Next: Model Distillation