LLMs produce free-form text by default. To build reliable applications, you need structured output: validated JSON with schemas, types, and retry logic. The extraction pipeline — prompt with schema, parse response, validate, retry — is the foundation of every production LLM application and AI agent.
You ask an LLM to analyze a customer review. It responds:
"The review was generally positive. The customer liked the design and performance but mentioned the price was a bit high."
That's a good answer. But how do you use it in code?
You'd need regex, NLP, or another LLM call to extract the sentiment, confidence, and reasoning. Every consumer of this response has to re-parse the same text differently.
Now consider this response:
"sentiment": "positive",
"confidence": 0.82,
"reasoning": "Customer praised design and performance, rated 4/5 stars"
}
Every consumer — your UI, your database, your analytics pipeline — can use this directly. This is structured output.
1. Free-Form vs Structured Output
LLMs naturally produce free-form text. Structured output constrains them to produce machine-readable data.
| Aspect | Free-Form Text | Structured JSON |
|---|---|---|
| Format | Natural language paragraph | JSON with defined keys and types |
| Parsing | Requires NLP, regex, or LLM | Direct json.loads() |
| Validation | No standard way | JSON Schema validation |
| Consistency | Varies every call | Same schema, same structure |
| Use in code | Hard to consume | Direct dictionary access |
| Error handling | Difficult to detect failures | Schema catches errors |
Structured output is not about limiting the LLM. It's about making its output reliable enough to build applications on.
2. JSON Schemas: Defining the Contract
A JSON schema defines exactly what the LLM output should look like: which fields, which types, which values are allowed.
SENTIMENT_SCHEMA = {
"type": "object",
"properties": {
"sentiment": {
"type": "string",
"enum": ["positive", "negative", "neutral", "mixed"]
},
"confidence": {
"type": "number",
"minimum": 0.0,
"maximum": 1.0
},
"reasoning": {
"type": "string",
"maxLength": 200
}
},
"required": ["sentiment", "confidence", "reasoning"]
}
The schema tells the LLM (and your validator) exactly what's expected. The enum constraint prevents the LLM from inventing values like "happy" or "frustrated" — it must choose from the defined list.
3. Validation: Catching Errors Before They Propagate
Even with a schema, LLMs can produce invalid output. Validation catches these errors:
errors = []
# 1. Check required fields
for field in schema.get("required", []):
if field not in data:
errors.append(f"Missing: '{field}'")
# 2. Check types
for name, prop in schema["properties"].items():
if name in data:
if not isinstance(data[name], type_map[prop["type"]]):
errors.append(f"'{name}' wrong type")
# 3. Check enums
# 4. Check numeric ranges
# 5. Check string lengths
# 6. Check array items
return ValidationResult(valid=len(errors)==0, errors=errors, data=data)
What Validation Catches
| Error Type | Example | Caught By |
|---|---|---|
| Missing field | {"sentiment": "positive"} |
Required check |
| Invalid enum | "sentiment": "happy" |
Enum check |
| Wrong type | "confidence": "high" |
Type check |
| Out of range | "confidence": 1.5 |
Range check |
| Too long | "reasoning": "..." (500 chars) |
MaxLength check |
| Bad array item | [{"name": "X"}] (missing type) |
Array item check |
4. Extracting JSON from Messy Output
LLMs don't always return clean JSON. They often wrap it in markdown code blocks or add explanatory text. Your application needs to handle all of these:
# Strategy 1: Direct parse
try: return json.loads(text)
except json.JSONDecodeError: pass
# Strategy 2: Extract from ```json code block
block = re.search(r'```(?:json)?\s*\n?(.*?)\n?```', text, re.DOTALL)
if block:
try: return json.loads(block.group(1).strip())
except json.JSONDecodeError: pass
# Strategy 3: Find first { ... }
brace = re.search(r'(\{.*\})', text, re.DOTALL)
if brace:
try: return json.loads(brace.group(1))
except json.JSONDecodeError: pass
# Strategy 4: Fix common LLM mistakes
cleaned = re.sub(r',\s*([}\]])', r'\1', text.strip())
try: return json.loads(cleaned)
except json.JSONDecodeError: pass
return None
The extraction handles four common LLM output patterns: direct JSON, markdown-wrapped JSON, JSON with surrounding text, and JSON with trailing commas.
5. Retry Logic: When Validation Fails
When the LLM produces invalid output, don't give up. Retry with a correction prompt:
for attempt in range(1, max_retries + 1):
# 1. Call LLM
raw = call_llm(prompt, schema)
# 2. Extract JSON
parsed = extract_json_from_text(raw)
if not parsed:
prompt = add_correction(prompt, "Return valid JSON only.")
continue
# 3. Validate
result = validate_json_schema(parsed, schema)
if result.valid:
return result.data # ✅ Success
# 4. Add error details to prompt for retry
prompt = add_correction(prompt, result.errors)
raise RuntimeError(f"Failed after {max_retries} attempts")
Retry in Action
| Attempt | LLM Output | Result |
|---|---|---|
| 1 | "sentiment": "happy" (invalid enum) |
❌ Validation failed → retry |
| 2 | "sentiment": "positive" (valid enum) |
✅ Valid → return result |
The retry prompt includes the validation error, guiding the LLM to fix its mistake:
6. Practical Use Cases
Sentiment Analysis
Return sentiment, confidence, and reasoning as typed fields — ready for dashboards and analytics.
Entity Extraction
Extract people, organizations, dates, and amounts from text as structured arrays:
"entities": [
{"name": "Apple Inc.", "type": "org", "value": "Apple Inc."},
{"name": "$94.8B", "type": "amount", "value": "94800000000"}
],
"summary": "Apple reports record Q3 revenue.",
"category": "business"
}
Tool Calls for AI Agents
When an AI agent needs to call a tool, structured output tells the system exactly which tool to call with which arguments:
"tool": "search",
"arguments": {"query": "Python asyncio tutorial", "max_results": 5},
"confidence": 0.92
}
This is how AI coding agents translate user intent into tool calls — the LLM outputs structured JSON that the agent runtime parses and executes.
For more on tool integration, see MCP vs APIs.
7. Production Patterns
| Pattern | When to Use | Implementation |
|---|---|---|
| Schema in prompt | Any provider | Include schema in system message |
| API response_format | OpenAI, Anthropic | Native structured output support |
| Validation + retry | All production systems | Validate, then retry with error details |
| Fallback to free-form | Non-critical output | Try structured, fall back to text parsing |
| Two-pass extraction | Complex schemas | First pass: free-form. Second pass: structure it. |
8. Common Mistakes
| Mistake | Consequence | Fix |
|---|---|---|
| No validation | Invalid data propagates silently | Always validate against schema |
| No retry logic | First bad output = application failure | Retry 2-3 times with error feedback |
Trusting json.loads alone |
Valid JSON, wrong types or values | Schema validation catches type/value errors |
| Overly complex schema | LLM struggles to follow | Start simple, add fields incrementally |
| Ignoring markdown wrapping | JSON parse fails on ```json |
Use multi-strategy JSON extraction |
9. FAQ
10. Practical Exercises
extract_json_from_text function and test it with these inputs: bare JSON, markdown-wrapped JSON, JSON with explanation text, and JSON with trailing commas. Which strategies work for each?
Continue Learning
Complete beginner tutorial with text summarizer How AI Coding Agents Work
The agent loop: plan, execute, observe Prompt Engineering Tutorial
Write prompts that produce reliable output Context Engineering Explained
Control what the model sees MCP vs APIs
Tool integration for AI agents JSON Formatter
Format and validate JSON output