Structured Output

What Structured Output Is

Structured output is a capability that forces an LLM to follow a predefined schema (usually JSON Schema) exactly, rather than emitting free-form prose. It goes well beyond simply asking "please answer in JSON" through the prompt; the point is to make it impossible for the model to generate a response that deviates from the schema in the first place. As a result, you get data that a program can parse immediately, free of missing keys, wrong data types, or broken JSON.

The concept emerged to connect LLM output reliably to downstream systems such as databases, APIs, and function calls. Processing a model's response in code requires the format to match precisely every time, yet models would historically tack on an explanatory sentence or drop a field, causing parse errors. Structured output removes that uncertainty, providing the foundation that raises the reliability of AI search, agent, and data-extraction pipelines.

Not the Same as SEO 'Structured Data'

The names are similar enough to invite confusion, but structured output is entirely distinct from what SEO calls structured data, the schema.org-based JSON-LD markup. Structured data is metadata embedded in HTML so that search engines can understand a page's content. Structured output, by contrast, is a capability that controls the format of the response the LLM itself produces. The two differ in purpose, where they operate, and who consumes them. The table below summarizes the distinction.

Aspect	Structured Output	Structured Data (Schema Markup)
Purpose	Force an LLM response into a specified format	Describe page content to search engines
Format	JSON Schema (the response body)	schema.org vocabulary as JSON-LD, Microdata, etc.
Location	The LLM API response	Markup inside the HTML document
Consumer	Downstream code, APIs, agents	Search engine crawlers such as Google and Bing
Typical example	OpenAI Structured Outputs, Gemini responseSchema	Rich result markup (star ratings, FAQ, products)

How It Works: Token Constraints, Not Prompt Requests

At the heart of structured output is constrained decoding. The defined JSON Schema is compiled into a grammar, and during the inference step where the model generates tokens one at a time, any token the schema does not permit is blocked. In other words, a format violation does not merely "happen rarely"—it becomes structurally impossible.

Anthropic's Claude documentation describes this process as "compiling your JSON schema into a grammar to constrain token generation during inference," explicitly noting that it does not rely on prompting. This is what separates JSON mode, which only guarantees "valid JSON," from structured output, which guarantees "exact schema adherence" as well.

Aspect	JSON Mode	Structured Outputs
Guarantees valid JSON	Yes	Yes
Guarantees schema adherence	No	Yes
Prevents missing required fields	No	Yes
Mechanism	Format steering (prompt)	Constrained decoding (token blocking)

JSON Schema Example

The foundation of structured output is JSON Schema. JSON Schema is a declarative language for defining the structure and constraints of JSON data, and the current recommended version is draft 2020-12. For instance, a schema that extracts user information might look like this.

{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "email": { "type": "string" },
    "plan_interest": {
      "type": "string",
      "enum": ["free", "pro", "enterprise"]
    },
    "demo_requested": { "type": "boolean" }
  },
  "required": ["name", "email", "plan_interest", "demo_requested"],
  "additionalProperties": false
}

In the OpenAI API you pass this schema in response_format, and you must enable strict for the schema to be enforced. OpenAI requires that every field be listed in required and that additionalProperties be set to false.

response_format = {
  "type": "json_schema",
  "json_schema": {
    "name": "user_info",
    "strict": true,
    "schema": { /* the schema above */ }
  }
}

In the Claude API you pass the same schema in output_config.format with type: "json_schema", and to strictly enforce even the arguments of a tool call, you add strict: true to the tool definition. Google Gemini distinguishes between responseSchema (schema enforcement) and response_mime_type (JSON mode), and defining your schema with Pydantic in Python or Zod in JavaScript works directly.

Evidence and Measured Data

The effectiveness of structured output is backed by official material from the providers.

• OpenAI: Announced that gpt-4o-2024-08-06 with Structured Outputs scored 100% on a complex JSON Schema adherence evaluation, whereas the older gpt-4-0613 managed under 40%. OpenAI recommends always using Structured Outputs over JSON mode whenever possible.
• Anthropic: Explains that Claude's structured output compiles the schema into a grammar to constrain token generation, so JSON.parse() errors do not occur and retries for schema violations are unnecessary. The compiled grammar is cached for 24 hours. That said, some JSON Schema features such as recursive schemas and lookahead, along with the number of tools and parameters, are subject to limits.
• Google: The Gemini documentation states that enforcing a schema means "eliminating the need for fragile output parsing," and presents data extraction, classification, and agent workflows as representative use cases.

Implementation Checklist

• Whenever possible, use structured output that enforces a schema rather than plain JSON mode.
• In your JSON Schema, list every field in required and set additionalProperties to false (OpenAI's recommendation).
• For fields that need an enumeration, narrow the set of valid values with enum to block invalid ones.
• Rather than writing the schema by hand, define it with Pydantic (Python) or Zod (JavaScript) to keep code and schema in sync.
• Handle exceptions by checking whether the response was a refusal or was truncated by a length limit (finish_reason).
• Some features such as recursive schemas and complex numeric constraints may be unsupported, so review each provider's schema limitations.

References

• OpenAI — Introducing Structured Outputs in the API
• OpenAI API Docs — Structured model outputs
• Anthropic Claude Docs — Structured outputs
• Google Gemini API — Structured output
• JSON Schema — What is JSON Schema? (draft 2020-12)