What are structured outputs in an LLM API?

Ask a model to "reply in JSON" and it usually will. Usually is the problem. Now and then it wraps the JSON in a sentence, drops a field or invents one, and the line of code that reads result["category"] fails in production. Structured outputs remove that gap: you send a JSON schema with the request, and the model's answer is made to fit it.

How it works

A model writes one TokenTokenThe small piece of text a language model reads and writes, often part of a word. Prices, limits and the context window are all counted in tokens. at a time, choosing from every token it knows. With structured outputs, the API turns your schema into a grammar and, at each step, only lets the model pick tokens that keep the answer valid against it. This is called constrained decoding. The model cannot open a field your schema does not have or close the object before a required field is written, because those tokens are not on offer.

That is why it is stronger than an instruction in the System PromptSystem PromptInstructions the builder of an AI app gives the model before you type anything. It sets the role, the tone and the rules, and it outranks what users say.. An instruction asks; the grammar enforces. It is also stronger than the older "JSON mode", which only promised valid JSON, not your fields.

What it looks like in code

In the Claude API you pass the schema in output_config, or in Python you give the SDK a Pydantic class and it builds the schema for you:

from pydantic import BaseModel
import anthropic

class Ticket(BaseModel):
    category: str
    order_id: str | None

client = anthropic.Anthropic()
response = client.messages.parse(
    model="claude-opus-5-5",
    max_tokens=4000,
    messages=[{"role": "user", "content": "Where is my parcel? Order 88231."}],
    output_format=Ticket,
)
print(response.parsed_output.order_id)

OpenAI's Responses API does the same with text.format set to a strict JSON schema, or client.responses.parse with a Pydantic class.

What it does not guarantee

The shape is guaranteed. The content is not. A schema can force a category field, but nothing in it makes the category right, so you still check answers with an eval set. 3 more things sit outside the guarantee:

• A refusal. If the model declines, the answer is a refusal, not your object. Check for it before you read fields.
• Running out of room. If the answer hits max_tokens, it stops mid-object. Check the stop reason, and leave room in max_tokens.
• A schema the grammar cannot express. Limits such as a minimum number or a maximum length are not enforced, so validate those in your own code. Recursive schemas are not supported at all.

In the Python SDK, a refusal or a cut-off answer makes messages.parse raise a validation error during the call, before you can read the stop reason. Catch it, or send the schema with messages.create and output_config, check that the stop reason is end_turn, and validate the text yourself.

The first request with a new schema is slower, because the grammar is compiled and then cached for 24 hours. A schema that changes on every request pays that cost every time.

Structured outputs or tools

If you want an answer in a fixed shape, use structured outputs. If you want the model to decide whether to call a function, give it a tool with a Function SchemaFunction SchemaThe description of a tool that a model reads before calling it, its name, what it does and a JSON Schema of its arguments. The model fills in the arguments; your code checks them and runs the function.; strict tool use applies the same grammar to the tool's inputs. An app can use both: tools while the model works, a schema for the final answer.