Service tiers (model APIs)

What are service tiers?

A service tier is how the provider serves your request, not which model answers it. Same model, same weights, same answer: what changes is where you sit in the queue and what the tokens cost. That is worth saying out loud, because service tier gets confused with model tier. Model tier is Haiku against Opus, the rung you pick off the size and price ladder. Service tier is that same model at list price, at a premium for speed, or at half price if you can wait.

You set it per request, in one field. OpenAI and Google take a service_tier value on the call, Anthropic takes the same field with two values, and on Microsoft Foundry it is a property of the deployment that a request can override. Your prompt, your model and your output stay as they were, which makes it the cheapest lever in the API.

The names do not line up across providers, and that is worth knowing before you compare two quotes. What OpenAI now calls Fast mode, Google calls Priority and Microsoft calls priority processing. What OpenAI and Google call Flex has no equivalent at Anthropic, whose Priority Tier is a purchased commitment rather than a per-request premium.

What each tier trades

• Standard. List price, best effort, and what you get if you send nothing. Microsoft says plainly that customers with high consistent volume on Global Standard can see greater latency variability, which is the honest description of every standard tier.

• Faster, for a premium. OpenAI renamed priority processing to Fast mode on 30 July 2026 and prices it at double the standard rate: GPT-5.6 Terra goes from 2 and 12 dollars per million input and output tokens to 4 and 24, for up to 2.5 times the speed on GPT-5.6 Sol. Google's Priority tier is 1.8 times standard. Microsoft publishes a latency target per model instead of a multiplier, such as 99 percent of gpt-5.6-terra requests above 70 tokens per second.

• Reserved capacity, for a commitment. You can also buy predictable latency up front: provisioned throughput units on Microsoft Foundry, Provisioned Throughput on Vertex AI, and Anthropic's Priority Tier, a number of input and output tokens per minute on one model version for 1, 3, 6 or 12 months at a target of 99.5 percent uptime. Check what is still on sale before you plan around it, because Anthropic has stopped selling new Priority Tier commitments.

• Cheaper, if you can wait a little. Flex is a normal synchronous call at half the price. OpenAI prices flex tokens at batch rates and still calls it beta with limited model availability. Google's Flex is also half of Standard, meant for work that tolerates the wait.

• Cheapest, if nobody is waiting at all. The batch tier is half price with results inside a window, and it is a different call shape: you submit a file of requests and collect a file of answers. Batch inference is a subject of its own; on this ladder it is the bottom rung.

Which tier a workload belongs in

One question sorts almost everything: is a person waiting for this answer right now, with the screen open? If yes, it belongs on standard, and it moves up only when you can say what the extra seconds are worth. If no, it belongs on flex or batch, and all that is left to decide is how long you can wait.

The mistake that costs real money in a small company is not picking the wrong tier, it is never picking at all. The interactive tier is the default, so the first prototype ran on it, and the nightly job someone added in March runs on it too, because that is what the wrapper function does. The classification run and the weekly eval sweep are usually the bigger half of the bill, and nobody is watching either of them.

One month of volume, split three ways

A distributor runs three things on GPT-5.6 Terra. Prices from OpenAI's pricing page on 4 September 2026, in dollars per million tokens: standard 2 and 12, flex and batch 1 and 6, Fast mode 4 and 24.

• A chat assistant on the website. 40,000 requests a month at 3,000 input and 400 output tokens, so 120 million and 16 million: 240 plus 192, or 432 dollars.

• A nightly job that sorts tickets and writes product descriptions. 300,000 requests at 1,200 and 150, so 360 million and 45 million: 720 plus 540, or 1,260 dollars.

• A weekly eval sweep. 20,000 requests at 2,000 and 300, so 40 million and 6 million: 80 plus 72, or 152 dollars.

Everything on standard comes to 1,844 dollars. Now sort it. The nightly job moves to batch at 360 plus 270, so 630. The eval sweep moves to flex at 40 plus 36, so 76. The chat stays where it is. The month lands at 1,138 dollars, and nothing about the prompts or the model changed.

The 706 dollars you freed up pays for the part where speed actually counts. Fast mode doubles the chat assistant to 864, so the month becomes 1,570, still under the 1,844 you were paying before. The work nobody watches paid for the speed on the work people do watch.

The interactive tier against the batch tier: who is waiting

On the interactive tier, a person is waiting. The answer has to arrive while somebody watches a cursor blink, so the provider keeps compute ready for your peak and prices that readiness into every token. Everything around the call assumes it: token streaming so the first words show up early, retries with backoff, a fallback model for when the provider is slow.

On the batch tier, a queue is waiting. The provider slots your work into its quiet hours and hands back half of what it saves. In exchange you give up the hour and keep only the window, and twenty-four hours is a target rather than a promise. So a nightly job on batch needs an answer for the morning it is not finished: yesterday's numbers with a note on them, or a smaller run on flex to cover the gap.

What to watch out for with service tiers

Your invoice will list the same model several times. One model at four prices means a bill showing GPT-5.6 Terra at four rates, which will not reconcile against any single price per million tokens. That is correct, not a billing error. Tag your calls by tier and by workload up front, because the invoice alone cannot tell you which job ran where.

A cheaper tier can refuse you, so write the fallback. OpenAI returns a 429 resource unavailable on flex when there is no spare capacity, and does not charge you for it. Timeouts get likelier too, so raise the client timeout. Decide per job whether you wait and retry or fall back to standard at full price. Microsoft is retiring the automatic flex fallback on Foundry from 25 September 2026, after which an unsupported model returns a 400 instead of quietly working.

A premium tier can quietly downgrade. OpenAI and Microsoft both say that traffic ramping too fast gets some fast requests served at standard speed and billed at standard rates, with the service_tier field in the response telling you which one you got. Microsoft names the trigger: more than 50 percent extra tokens per minute inside fifteen minutes. Log the field that came back, not the parameter you sent.

Tiers multiply with caching and routing. Prompt caching cuts the repeated front of a prompt to a fraction of the input price, and model routing sends each request to the right rung of the size ladder. The tier is a percentage on whatever those two left you with, so pick the model first, cache what repeats, and set the tier last. OpenAI states that flex tokens still get the caching discount on top.