Reasoning effort

What is reasoning effort?

Reasoning effort is a request-level setting that decides how much internal work a model does before it writes its answer. At a low setting the model responds almost immediately. Raise it and the model first works through the problem privately, testing an approach, backing out of it, trying another, and only then produces the reply you see.

That private work is made of ordinary tokens. The model is still predicting one token at a time; it is simply allowed to spend a few thousand of them on itself first. Vendors call these reasoning tokens or thinking tokens, and most APIs do not return them to you. OpenAI's reasoning guide is direct about it: reasoning tokens are not visible via the API, and they occupy space in the model's context window and are billed as output tokens.

Think of it as the difference between answering a question in a meeting and asking for five minutes with a whiteboard first. The whiteboard usually produces a better answer. It also costs five minutes, and you pay for the whiteboard.

How each provider exposes the setting

The three large providers agree on the mechanism and disagree on the dial, so nothing you tune on one carries over to another.

OpenAI: named levels

OpenAI uses a reasoning.effort parameter with named levels. The documented values run from none and minimal through low, medium and high, up to xhigh and max, with support depending on the model you call. The guide describes the trade in one line: lower effort favours speed and lower token usage, while at higher effort the model thinks more completely. Azure OpenAI exposes the same parameter, and Microsoft's docs note that a higher effort setting means the model spends longer on the request and generally produces more reasoning tokens.

Defaults are not stable across model generations. Microsoft Learn points out that some newer models default their effort to none, so an application that relied on an older default and never set the parameter can quietly change behaviour after a model upgrade.

Anthropic: a token budget

Anthropic asks for a number instead of a level. You enable extended thinking by passing a thinking object with budget_tokens, which sets the maximum number of tokens Claude can use for its internal reasoning. The budget must be lower than max_tokens for the request, so raising it means raising the ceiling for the whole response.

A budget is a limit, not a quota to be spent. Anthropic notes that Claude may not use the entire budget allocated, especially at ranges above 32k, which means doubling the number does not reliably double the thinking you get for it.

Google: thinking levels with a dynamic default

Gemini models reason by default and adjust how much on their own, which Google calls dynamic thinking. A thinking_level parameter overrides that with minimal, low, medium or high, again depending on the model. Some models ship with thinking on and some ship with it off, so the default is a per-model fact rather than a platform-wide one.

What the hidden tokens cost

This is the part that surprises people on the first invoice. Reasoning tokens are billed as output tokens on every major platform, whether or not you ever see them. Google states it plainly: when thinking is turned on, response pricing is the sum of output tokens and thinking tokens.

Summarised thinking does not change the arithmetic. Anthropic returns a summary of Claude's thinking rather than the raw text, and bills for the full thinking tokens generated by the original request, not the summary tokens. The docs go further: switching the thinking blocks off entirely reduces latency, not cost. Google says the same about its thought summaries, where pricing is based on the full thought tokens the model needs to generate.

Put numbers on it. A ticket classifier that returns a single word answer produces maybe five visible output tokens. Run the same call at a high effort setting and the model might spend two thousand reasoning tokens deciding between two obvious categories. You are billed for roughly 2,005 output tokens instead of 5, for an answer that was never in doubt. Multiply that by every ticket that arrives in a month and the bill stops looking like a rounding error.

There is a second cost that is easier to miss. Because reasoning tokens sit inside the context window, a long thinking pass eats room that your prompt and your answer also need. OpenAI recommends reserving at least 25,000 tokens for reasoning and outputs when you start experimenting with these models.

Reasoning effort versus chain-of-thought prompting

Chain-of-thought prompting is a writing technique: you word the prompt so the model works step by step in its visible answer. Reasoning effort is a parameter: you leave the prompt alone and tell the platform how much private thinking to allow before the answer starts.

The distinction matters in practice. Chain-of-thought output lands in the response, so you can read it, log it and show it to a reviewer. Reasoning tokens usually do not, so you get the benefit without the audit trail. And because reasoning models already think before they answer, adding "think step by step" to a prompt on top of a high effort setting mostly buys you a longer, more verbose reply.

The comparison to temperature is closer. Both are runtime knobs on the same call, both are frequently tuned before anyone checks whether the prompt is the real problem, and neither adds knowledge the model does not have. Temperature changes which token gets picked. Reasoning effort changes how many tokens the model spends before it starts picking the ones you see.

What to watch out for with reasoning effort

Maximum by default
The most common mistake is setting the highest level once, everywhere, and never revisiting it. Extraction, classification, routing and formatting are tasks with one right answer and no deliberation to do. They get slower and dearer at high effort, and no better.

Assuming more effort means a better answer
The relationship flattens. Past a certain point the model has already found its approach and the extra budget goes into rechecking work it had right the first time. Treat effort as something you measure on a test set, the same way you would treat any other parameter, rather than a quality slider you push to the right.

Latency in interactive products
A chat interface where every reply takes twenty seconds is a worse product than one that answers in two, even when the slow answers are marginally better. Route by task instead: low effort for the common path, high effort for the handful of requests that genuinely need it.

Budget surprises at scale
Reasoning token counts vary from one inference call to the next, because the model decides how much of the allowance to use. A workload that averages fine can still produce individual calls that run far longer than the average. Log the reasoning token count that the API returns alongside your normal usage metrics, so the number is visible before the invoice is.