Reasoning model

What is a reasoning model?

A reasoning model is a language model that works through a problem internally before it writes its answer. Instead of producing the reply straight away, it first generates a stretch of private thinking: planning an approach, weighing alternatives, catching its own mistakes, and only then answering.

The useful intuition is that every token is room to compute in. An ordinary model has to commit to the first word of its answer immediately, which means it has almost no room to work out anything hard before it starts. A reasoning model gives itself that room first, and then answers from a position where it has already done the work.

You will also see this called extended thinking or adaptive thinking, depending on the provider. It is the same idea under different names, and the trade behind it is always the same one: more thinking buys quality and costs time and money.

Reasoning tokens, and why they show up on your bill

The thinking is made of tokens like everything else, and there are three things about them worth knowing.

You do not see them. The internal reasoning is not returned to you as normal text. What comes back is the answer, and depending on the provider a summary of the thinking.

You pay for them as output. Reasoning tokens are billed at the output rate, which is the expensive one. This is how a model that gives you a three-sentence answer can cost more than one that writes several pages: most of what you paid for was invisible.

They take up room in the context window. The thinking has to fit alongside your prompt and your answer. On a hard problem a model can generate anywhere from a few hundred to tens of thousands of reasoning tokens, so on a long conversation with a large prompt this is a real constraint and not a footnote.

The API tells you how many reasoning tokens a request used, and that number is the one to watch when you are trying to work out where your bill is going.

How much thinking do you allow?

You get a control over the depth, and it is worth using deliberately.

Some providers expose it as an effort level, running from no thinking at all up to maximum. Others expose it as a thinking budget in tokens, where you set a target for how much the model may spend before it starts answering. Anthropic sets the minimum budget at 1,024 tokens and rejects anything smaller.

Two things are worth knowing about tuning it. First, the returns diminish, and where they start diminishing depends on the task, so this is something you test rather than reason about. Second, a budget is usually a target and not a hard stop. The model can finish thinking well before it has used the budget, so a generous budget does not automatically mean a generous bill.

The practical approach is to start low on simple tasks and raise it until quality stops improving, and to start high on genuinely hard tasks and lower it until quality starts dropping. On anything that matters, test both directions rather than picking a setting once and leaving it.

When a reasoning model is worth it

Providers are reasonably consistent about where this helps: complex problem solving, coding, scientific reasoning, and agentic work that runs over several steps and uses tools. Those all share a property. There is a right answer, getting there takes several steps, and a mistake early on ruins everything after it.

They are equally consistent about where it does not help. Anything where speed is the point is a bad fit. A voice assistant that has to reply immediately, a fast lookup, a simple classification, a routing decision: all of those are better on a standard model, because the thinking adds seconds without adding correctness.

There is a middle case worth naming. On agent work, thinking often pays for itself in a way that is not obvious from the price per call. An agent that plans properly before it starts calling tools makes fewer wrong tool calls, and every wrong tool call costs you a round trip plus the tokens to read the useless result. Comparing purely on price per call misses that.

What to watch out for with reasoning models

Latency is the first thing your users notice. A model that thinks for fifteen seconds before answering feels broken behind a chat window unless you show that something is happening.

The thinking is not an explanation. What you get back, when you get anything, is a summary of an internal process. It is a useful debugging aid. It is not an audit trail, and it should not be presented to a customer or a regulator as the reason a decision was made.

Thinking does not fix missing information. A model that does not have the fact will reason its way to a confident wrong answer just as readily as it reasons its way to a right one. If the answer lives in your documents, retrieval is the fix, not more thinking.

Budget and cost do not move together in a straight line. Doubling the thinking budget does not double your bill, because the model often stops early, and it does not double your quality either. Measure both instead of assuming.

Model generations change the controls. Providers keep changing how thinking is configured, from fixed budgets towards the model deciding for itself how much to think. Anything you hard-code today is worth revisiting when you move to a newer model.