Structured output

What is structured output?

Structured output means making a model return data in a fixed shape that software can read directly, instead of a paragraph of prose. In practice that shape is almost always JSON with named fields, set types, and a list of allowed values.

Free text is fine when a person reads the answer, but it breaks down when the next step is code. If a workflow has to act on an LLM reply, you want a category field and a confidence field back every time, not a sentence you have to parse.

Asking for JSON versus enforcing a schema

There is a real difference between a prompt that says "reply in JSON" and an API that guarantees the reply fits your schema. A line in a prompt template is a request: the model usually complies, but it can still drop a field or return a value that was never allowed.

Structured output closes that gap while the answer is generated. You hand the API a JSON schema, and platforms such as OpenAI and Azure OpenAI compile it into a grammar that constrains decoding, so at each step the model can only pick a token that keeps the output valid against the schema. OpenAI states the model "will always generate responses that adhere to your supplied JSON Schema", and Anthropic gives the same promise through strict tool use. A finished response parses and fits the shape you defined.

A schema names the fields, their types, and the allowed values. Enums matter, since they force the model to pick from a fixed set instead of writing its own labels:

{
  "type": "object",
  "properties": {
    "category": { "type": "string", "enum": ["bug", "billing", "other"] },
    "confidence": { "type": "number" }
  },
  "required": ["category", "confidence"],
  "additionalProperties": false
}

Where you use structured output

• Extraction. Turn an invoice email into supplier, amount, due_date, and po_number that flow straight into your system.

• Classification. Route a ticket into one of five categories with a confidence score, so a workflow branches on a field instead of a sentence.

• Agent tool calls. When an AI agent uses tool use (function calling), it must emit arguments that match the tool schema exactly, and structured output makes that call dependable.

What to watch out for with structured output

Valid is not the same as correct. The schema controls the shape, not the truth of the values. Ask a weather tool for a forecast without naming a city and the model may still fill location with "New York, NY": schema-valid and invented. Checking that the values are right stays your job, which is where validation and evals come in.

The guarantee assumes the response finishes. A refusal or the token limit can end a reply before the schema is satisfied, so handle the case where nothing valid comes back.

Keep the schema lean and legal. Structured output supports only a subset of JSON Schema. Every object needs additionalProperties set to false and every field must be required, with optional fields modelled as a type that also allows null. Fewer fields also means less for the model to get wrong.