Structured Outputs: Getting JSON You Can Trust From an LLM

Asking a model to “reply with JSON only” works about 95% of the time, which is exactly the worst possible reliability: often enough to ship, rarely enough to page you. Structured outputs replace that hope with a schema the API enforces. This is how to use them properly, and what they still do not guarantee.

Why “reply with JSON” is not a contract

The failure modes of prompt-only JSON are tediously familiar: a markdown fence wrapped round the object, a friendly sentence before it, a trailing comma, a field renamed from severity to priority, an enum value that was never in your enum. Every one of those produces a stack trace three layers away from the cause.

Regex-stripping fences and retrying on JSONDecodeError is the usual patch. It costs you an extra request every time it fires, and it does nothing about the renamed field. Constrain the output instead.

Structured outputs with a Pydantic model

The neatest route in Python is to declare the shape as a Pydantic model and use the SDK’s parse helper. You get a validated instance back, not a string to inspect.

from typing import Literal, Optional
from pydantic import BaseModel, Field
import anthropic

client = anthropic.Anthropic()

class Triage(BaseModel):
    summary: str = Field(description="One sentence, no more than 20 words.")
    severity: Literal["low", "medium", "high"]
    affected_service: Optional[str] = Field(
        description="Service name if stated, otherwise null. Never guess."
    )
    needs_human: bool

response = client.messages.parse(
    model="claude-opus-5",
    max_tokens=4096,
    system="You triage inbound support tickets.",
    messages=[{"role": "user", "content": raw_ticket}],
    output_format=Triage,
)

triage = response.parsed_output
print(triage.severity, triage.needs_human)

response.parsed_output is a real Triage instance. If you already use Pydantic at your HTTP boundary — and if you are on FastAPI you do, as I noted when digging into what changed in FastAPI 0.140 — the same models can serve both ends, so the shape the model produces is the shape your endpoint returns.

The raw schema version

Without Pydantic, pass a JSON Schema on the request. Watch the parameter name — the parse helper takes output_format, while messages.create() takes output_config with a format inside it. Mixing them up is a confusing 400.

import json

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=4096,
    messages=[{"role": "user", "content": raw_ticket}],
    output_config={
        "format": {
            "type": "json_schema",
            "schema": {
                "type": "object",
                "properties": {
                    "summary": {"type": "string"},
                    "severity": {"type": "string", "enum": ["low", "medium", "high"]},
                    "affected_service": {"type": ["string", "null"]},
                    "needs_human": {"type": "boolean"},
                },
                "required": ["summary", "severity", "affected_service", "needs_human"],
                "additionalProperties": False,
            },
        }
    },
)

text = next(b.text for b in response.content if b.type == "text")
data = json.loads(text)

additionalProperties: false and a complete required list are not optional decoration — they are what makes the constraint strict. One incompatibility to know about: this cannot be combined with document citations, which returns a 400.

Strict tool inputs are the other half

Structured outputs constrain what the model says to you. If it is calling your tools, you want the arguments constrained too. Same idea, different place — strict: true sits on the tool definition itself.

tools = [{
    "name": "create_incident",
    "description": "Open an incident in the on-call system.",
    "strict": True,
    "input_schema": {
        "type": "object",
        "properties": {
            "title": {"type": "string"},
            "severity": {"type": "string", "enum": ["low", "medium", "high"]},
        },
        "required": ["title", "severity"],
        "additionalProperties": False,
    },
}]

Streaming partial JSON

Here is the gotcha that catches everyone. If you stream a structured response, the text arriving mid-stream is a valid prefix of JSON, not valid JSON. {"summary": "Disk fu parses fine as far as the model is concerned and not at all as far as json.loads is concerned.

Do not parse per chunk, and do not reach for a partial-JSON library unless you genuinely need to render fields as they arrive. Accumulate, then parse once.

with client.messages.stream(
    model="claude-opus-5",
    max_tokens=8000,
    messages=[{"role": "user", "content": raw_ticket}],
    output_config={"format": {"type": "json_schema", "schema": schema}},
) as stream:
    for text in stream.text_stream:
        progress(text)          # display only — never json.loads here

    final = stream.get_final_message()

data = json.loads(next(b.text for b in final.content if b.type == "text"))

The same trap exists on the tool side, and it gets worse if you enable fine-grained tool streaming (eager_input_streaming: true on the tool definition), which pushes argument fragments out sooner. Also check stop_reason before parsing: if it is max_tokens, your JSON is truncated and no schema can save it.

What a schema still cannot promise

Structured outputs guarantee shape. They do not guarantee truth. A perfectly valid object can contain a confidently invented service name.

• Prefer enums to free strings wherever the set is closed — that turns “wrong” into “impossible”.
• Give the model a way to say “I don’t know”. A nullable field with “never guess” in its description beats a required string it will fill in regardless.
• Keep business rules in code. If severity high must set needs_human, assert it after parsing rather than describing it in the schema.
• Flat schemas beat deeply nested ones. Accuracy falls off well before the schema stops validating.

What I’d do

Use the Pydantic parse helper for anything that feeds code — it is the shortest path from model to validated object, and the model doubles as your API schema. Reach for the raw JSON Schema form only when the shape is dynamic. Whatever you choose, never parse a stream mid-flight, use enums aggressively, and keep every rule that actually matters in Python where you can test it.