Since Ollama 0.3.0, passing a JSON schema to the format parameter eliminates this problem at the root. The model's inference itself is constrained by the schema, so no code fences, no explanatory text, no mid-thought artifacts. Just parseable JSON.

I ran these tests locally with Gemma4 and Ollama 0.30.7 to see how well it holds up in practice.

Why LLM Response Parsing Is Tricky

The most common problem when running Ollama locally — without a cloud LLM API — is JSON parsing. Two reasons.

First, text generation models are trained toward "natural text." Even if you ask for JSON only, they'll often wrap it in

json ...

json
Input: 'Give me 3 Python tips as JSON with keys: tips (array), difficulty (1-5)'
Model output (no format parameter):


```json
{
  "tips": [
    "Master the fundamentals first...",
    ...
  ]
}

JSON parse: FAILED

Python's `json.loads()` can't handle the markdown wrapper. The "JSON only" instruction is unreliable in production.

Second, speed. I measured the same query both ways: 32 seconds without structured output, 5 seconds with it. More on why below.

## How the Ollama format Parameter Works

Ollama's `/api/generate` endpoint has a `format` field. Pass a JSON schema object and Ollama applies **constrained decoding** during inference.

python
import json
import urllib.request

def ollama_structured(prompt, schema, model="gemma4:e4b"):
payload = {
"model": model,
"prompt": prompt,
"format": schema, # ← pass JSON schema object directly
"stream": False,
"options": {"temperature": 0}
}
data = json.dumps(payload).encode()
req = urllib.request.Request(
"http://localhost:11434/api/generate",
data=data,
headers={"Content-Type": "application/json"}
)
with urllib.request.urlopen(req, timeout=60) as resp:
result = json.loads(resp.read())
return result["response"]

Constrained decoding sets the probability of any token that would violate the schema to zero at each generation step. So even if the model "wants" to generate a markdown fence, the schema makes it physically impossible. That's also where the speed gain comes from — the model doesn't waste tokens on formatting decisions.

Here are the measured numbers:

bash

Direct measurement (Ollama 0.30.7, Gemma4:e4b, MacBook)

Same prompt, with and without format

With structured output:
Structured: {"tips": ["Understand the concept of indentation...", ...], "difficulty": 2}
Time: 4.99s
JSON parse: SUCCESS

6.4x difference. Local LLMs are already slow, and adding unreliable parsing on top makes the whole pipeline feel worse.

## Wiring Pydantic Models

Writing JSON schema objects by hand is tedious. With Pydantic models, `model_json_schema()` generates the schema automatically.



```python
from pydantic import BaseModel
from typing import List, Dict, Any, Literal

class CodeReview(BaseModel):
    severity: str  # "critical", "warning", "info"
    file: str
    line: int
    message: str
    suggestion: str

class ReviewResult(BaseModel):
    total_issues: int
    critical_count: int
    reviews: List[CodeReview]

# Pydantic → JSON schema, automatically
schema = ReviewResult.model_json_schema()

raw = ollama_structured(prompt, schema)

# Parses and validates in one step
result = ReviewResult.model_validate_json(raw)

model_validate_json parses the JSON string and runs Pydantic validation simultaneously. If severity gets an integer or line gets a string, it throws ValidationError. Catching that and retrying with a modified prompt is the common pattern in real agents.

Actual output from the code review test:

=== Code Review Output ===
Total issues: 3
Critical: 2
  [CRITICAL] process_user_data:2 - SQL Injection Vulnerability (Direct String Formatting)
  [CRITICAL] process_user_data:3 - Storing Passwords in Plain Text (Data Leakage)
  [HIGH] process_user_data:4 - Potential Unused/Incomplete Database Interaction

total_issues: 3 and critical_count: 2 come in as integers. if result.critical_count > 0 branches safely.

Practical Pattern: Agent Tool Dispatch

The strongest use case for structured output is an agent deciding which tool to call next. You pass the tool list and current situation, and get back a type-safe tool call selection.

from typing import Literal, Dict, Any

class ToolCall(BaseModel):
    tool_name: Literal["web_search", "read_file", "write_file", "execute_code"]
    parameters: Dict[str, Any]
    reasoning: str

schema = ToolCall.model_json_schema()

user_task = "Find the current Bitcoin price and save it to btc_price.txt"
prompt = f"""You are an AI agent. Decide which tool to call next.
Available tools: web_search, read_file, write_file, execute_code
Task: {user_task}
Choose ONE tool call."""

raw = ollama_structured(prompt, schema)
tool_call = ToolCall.model_validate_json(raw)

print(f"Tool: {tool_call.tool_name}")
print(f"Params: {tool_call.parameters}")

=== Agent tool dispatch ===
Tool: web_search
Params: {'query': 'current Bitcoin price'}
Reasoning: The task requires finding real-time information...

Dispatch: OK (type-safe)

Because tool_name is typed as Literal["web_search", "read_file", ...], tool_call.tool_name is always one of those four values. If the model invents a nonexistent tool name, Pydantic throws ValidationError. The if tool_call.tool_name == "web_search" branch is safe to write.

This is architecturally the same as function calling in cloud APIs. Comparing it with Claude Agent SDK's Tool Use patterns shows an interesting design difference: cloud LLMs handle tool selection natively at the model level, while local Ollama needs an explicit JSON schema + Pydantic validation layer.

Gemma4 and Schema Complexity: Limitations I Found

Honestly, it doesn't work perfectly in every case. Testing with Gemma4:e4b (4-bit quantized, 4B parameters), I found a few real constraints.

Deeply nested schemas. JSON schemas nested 3+ levels deep (List[Dict[str, List[BaseModel]]]) sometimes return empty arrays at intermediate levels. The 12B model (gemma4:12b-it-qat) reduces this, but doesn't eliminate it. This is a fundamental limitation of the model's context handling.

Optional field handling. Fields declared as Optional[str] sometimes get filled with empty string "" instead of null. Pydantic validation passes, but semantics differ. You need @validator post-processing.

Schema size. A large Pydantic model's JSON schema can reach hundreds of tokens. That occupies context window space, reducing the room available for the actual prompt. Complex schemas need stronger models.

Once you've deployed Ollama as an API server (covered in the Ollama FastAPI production guide), switching models at runtime based on schema complexity becomes a viable optimization.

Pattern Reference: When to Use What

I think of this as a switch that moves JSON parse reliability from "unreliable" to "near 100%." There was a time I was appending "JSON only please" to every prompt and hoping for the best. Measuring the actual difference made clear how fragile that approach was.

Copy-Paste Starter Code

import json
import urllib.request
from typing import List, Optional, Dict, Any
from pydantic import BaseModel

def ollama_structured(prompt: str, model_cls: type[BaseModel], 
                      model: str = "gemma4:e4b") -> BaseModel:
    """
    Helper that combines Ollama structured output + Pydantic validation.
    """
    schema = model_cls.model_json_schema()
    payload = {
        "model": model,
        "prompt": prompt,
        "format": schema,
        "stream": False,
        "options": {"temperature": 0}
    }
    data = json.dumps(payload).encode()
    req = urllib.request.Request(
        "http://localhost:11434/api/generate",
        data=data,
        headers={"Content-Type": "application/json"}
    )
    with urllib.request.urlopen(req, timeout=120) as resp:
        result = json.loads(resp.read())
    return model_cls.model_validate_json(result["response"])


# Usage example
class SentimentAnalysis(BaseModel):
    sentiment: str       # "positive", "negative", "neutral"
    confidence: float    # 0.0 ~ 1.0
    key_phrases: List[str]

result = ollama_structured(
    "Analyze sentiment: 'This new MacBook is amazing but too expensive'",
    SentimentAnalysis
)
print(f"Sentiment: {result.sentiment} ({result.confidence:.0%})")
print(f"Key phrases: {result.key_phrases}")

What to Try Next

This only covers the simplest cases. A real agent needs a bit more.

Retry logic. When Pydantic ValidationError fires, retry with a slightly modified prompt — ideally including the error message. Models often self-correct when they can see why they were wrong.

Streaming. With stream: true, you can receive the JSON incrementally as it generates. Pair with a streaming JSON parser like ijson for memory-efficient handling of large responses.

Model switching. Route simple extractions to gemma4:e4b (fast) and complex nested schemas to gemma4:12b-it-qat (accurate) at runtime. Structuring an entire agent with Pydantic AI shows how to abstract this decision to the framework level.

If you're already running a Gemma4-based agent locally, adding the format parameter today is a one-line change with a measurable reliability improvement. Especially anywhere in the agent loop where an invalid response immediately causes a downstream error.