Managed LLM service

What is a managed LLM service?

A managed LLM service is an arrangement where you buy access to somebody else's language model through a business contract with a cloud provider or a model vendor. You get an endpoint, a key and an invoice. The weights stay on their machines, the version is theirs to change, and running GPUs is not your problem.

What you are buying is not a model. It is an endpoint with paperwork behind it. The same weights are usually reachable through a consumer app and through three or four business routes, and what separates them is what the contract says about your text, not what the model can do.

Two neighbours mark the edges. A consumer subscription is the same technology sold to a person, with terms written for a person. A local LLM is the opposite end, where the model runs on hardware you control and no contract sits between you and your data. At the volumes an SME runs, the managed route is almost always cheaper in money alone, so cost rarely decides this. What decides it is whether a class of your data may leave the building, and whether you can live with the provider changing the model when it suits them.

What you are actually buying

Four things arrive with a managed LLM service, and only the first is technical.

An endpoint, with whatever availability promise is genuinely on offer. Read it rather than assume it. Anthropic's documentation describes its standard tier as best-effort availability, and the Priority Tier that targeted 99.5 percent uptime is no longer available for purchase. Reaching the same model through a cloud provider puts the endpoint under that provider's service terms instead.

The paperwork. A data processing agreement, a written retention position, a sub-processor list you are told about when it changes, and a statement about which region processes your requests. For most business buyers those four documents are the actual product. Whether the provider trains on your text and how long it keeps it are the two questions the zero data retention entry works through in detail.

Administration. Keys per team rather than one shared secret, usage you can attribute to a person or a project, spend limits and an audit trail.

An invoice that survives scrutiny. A company name, a VAT number, line items per model and per project. A charge on a colleague's personal card is not a record for work that touches client data.

What you are not buying is control of the version. That stays with the provider, and it is the price of not owning the hardware.

The same model through four doors

Take one model family, Anthropic's Claude, and look at the routes a company can buy it through. The capability is broadly the same. The contract is not. This is what the vendors' own documentation said in September 2026.

• Anthropic's own API. Anthropic is the data processor. You can pin inference with the inference_geo parameter, but only to us or global, and US-only inference costs 1.1 times the standard rate on Claude 4.6 and later. Zero data retention is granted per organisation through the sales team, and does not cover the Fable and Mythos 5 models, which require 30-day retention.

• Amazon Bedrock. AWS is the data processor, not the model vendor. AWS documents zero operator access and zero data retention as the default, and says model providers have no access to Bedrock logs or to customer prompts and completions. Retention is a mode you set per account or per project and can force organisation-wide with a service control policy. Claude Fable 5 and 5.1 are the named exception: all traffic is kept for up to 30 days inside the AWS boundary, and the model turns unavailable if your setting is stricter than it requires.

• Microsoft Foundry. Prompts and completions are not used to train the models, and processing stays in the geography you name unless you pick a Global or DataZone deployment. A DataZone deployment created in an EU member state may be processed in that or any other EU member state. For deployments in the European Economic Area, the Microsoft employees authorised to review flagged content are located in the EEA. Turning that storage off is an application, not a setting.

• Microsoft 365 Copilot. Here Anthropic is a Microsoft sub-processor, so Microsoft's Product Terms and Data Protection Addendum cover the use. Microsoft enables these models by default in the commercial cloud but not in the EU, EFTA or the UK, where an admin has to opt in, and they are currently excluded from the EU Data Boundary. Some models come instead as "Anthropic models with Data Retention", where Anthropic acts as an independent processor under its own terms and keeps most inputs and outputs for up to 30 days, flagged content for up to two years.

One model family, four contracts, three different answers to the question of who your processor is. On a cloud route the cloud provider is your processor and the model vendor sits underneath as a sub-processor; on a direct route the model vendor is your processor. That decides which company you have a claim against and whose sub-processor list you need to watch. So if a supplier tells you they use Claude, you have learned almost nothing about your own exposure until you know which door they came through.

Region deserves the same care. A parameter you can set is not the same as a boundary you cannot cross. Anthropic lets you fix the allowed geos per workspace so a stray request cannot leave, and Microsoft lets you pick a deployment type that constrains processing. Ask which mechanism enforces it, not only which value is set today.

The provider changes the model under you, on their calendar

Model versioning and deprecation covers what the model vendors promise on their own APIs. The point here is that a managed route adds a second clock, and can move you without asking. Microsoft sets a retirement date programmatically when a model reaches general availability in Foundry, 18 months out. Models from Anthropic, DeepSeek, Fireworks and Mistral AI run on a 12-month lifecycle instead. At retirement every inference request returns 410 Gone. Subscription owners with active deployments get at least 60 days' notice, and Microsoft is blunt about extensions: retirement dates are not extendable.

The part that catches people is the automatic upgrade. On Standard, Global Standard and Data Zone Standard deployments Microsoft migrates you to the replacement model region by region, unless you set versionUpgradeOption to NoAutoUpgrade, in which case the deployment stops working at retirement. Provisioned deployments are never auto-upgraded and have to be moved by hand. So the default on the cheapest deployment type is that the model behind your production workflow changes without a deployment on your side, and the alternative default is that it stops. Neither is wrong, but both need a name against them: somebody who reads the provider's mail, a test set of your own examples, and every model identifier in configuration where you can find it in a minute.

Managed service versus consumer subscription: who carries the obligation

The dimension that matters is not the price and not the model. It is who answers for the personal data in the prompt, and what you can show when somebody asks.

Under the GDPR you are the controller for the client names, personnel files and invoices your people paste into a chat box, and that does not change with the tool. What changes is whether the party processing it for you is bound by a written agreement with your company. On a managed service they are: the processing agreement is signed by the organisation, the retention position is documented, the region is a setting somebody configured, and usage is tied to identities you manage. Under the AI Act you are the deployer of the system, and the same paperwork is what shows you took that role seriously.

With a consumer subscription almost none of that exists in your name. The terms were accepted by an individual, often with a personal e-mail address. You cannot produce a processing agreement, you cannot say which region handled the request, you cannot list who used it last month, and when that person leaves you cannot retrieve their history or delete it. The provider may well treat the data properly, but you have no way to show it, which for an audit or a client questionnaire amounts to the same thing as having nothing. That gap is where shadow AI lives: a negotiated agreement covering a workspace nobody logs into, while the real work happens in personal accounts.

What to watch out for with a managed LLM service

The lock-in is not the API. Swapping one chat completions endpoint for another is a day of work, and an LLM gateway makes it a configuration change. What holds you is everything built around one model's behaviour: prompts tuned to its habits, parsers shaped to its formatting, tool definitions it handles well, and an evaluation set that only tells you whether this model is good. Rebuilding that is the migration cost, and it grows quietly.

A budget alert is not a spend limit. An Azure Cost Management budget notifies you when spending reaches the amount you set; it does not stop anything. Anthropic's Enterprise plan does stop, with a monthly limit per member that resets at 00:00 UTC on the first of the month and an increase the member has to request from an admin. Find out which of the two you have before you let an agent loop unattended.

Read the terms for the tier you are on, not for the company. The same vendor sells opposite terms on a free quota and a paid one, and a key from the wrong side looks identical in your code.

Stateful features move you out of the strict arrangement. Anything that remembers between calls has to write something down, so once you add memory, stored conversations or long agent sessions, the retention answer you were given for plain calls may no longer apply.

A sub-processor list changes. Model vendors appear underneath products you already pay for, switched on by default in one region and off in another. Ask how you are told, and put that notification on somebody's desk rather than in a shared mailbox.