Guaranteed Structured Outputs & Schema Validation

TL;DR: Asking an LLM in plain text to "return valid JSON" works 80% of the time. The other 20%, it wraps output in markdown fences, hallucinates keys, or adds conversational commentary that crashes json.loads(). In this post, we explore Forced Function Calling and strict JSON schemas in Gemini to guarantee 100% schema-valid output.

📦 Interactive Notebook: Test and run this code live on GitHub: 07_structured_outputs.ipynb in the ai-architect-foundation-gemini repository.

What You'll Learn

Why prompt-based JSON generation is fragile, how to force a model into structured output mode using ToolConfig with mode="ANY", and how to extract strongly typed objects directly into your application.

diagram

Rendering diagram…

Why Asking for JSON in Prompts Fails

In naive LLM implementations, developers prompt the model: "Extract the order number and reason from the message and return ONLY JSON."

In production at scale, this fails predictably:

Markdown Fences: The model returns ```json\n{"order_id": "123"}\n``` instead of raw JSON.
Conversational Prefaces: The model prepends "Sure, here is your JSON object:".
Key Drift: It returns "orderNumber" instead of "order_id".
Missing Nullable Fields: It omits fields entirely rather than setting them to null.

Any of these subtle deviations will throw a json.JSONDecodeError or KeyError in your backend services.

The Solution: Forced Tool Calling (Structured Output)

Instead of treating structured extraction as text generation, we configure Gemini to treat the schema as a strict form that must be filled.

By configuring types.ToolConfig with FunctionCallingConfig(mode="ANY", allowed_function_names=[...]), the model is prohibited from returning text and is forced to call the specified schema tool.

python

from google import genai
from google.genai import types

client = genai.Client()
model = "gemini-2.5-flash"

# 1. Define the Extraction Schema as a Tool
extract_schema = types.FunctionDeclaration(
    name="extract_support_ticket",
    description="Extract structured customer support ticket details.",
    parameters=types.Schema(
        type="OBJECT",
        properties={
            "customer_name": types.Schema(type="STRING", nullable=True),
            "order_id": types.Schema(
                type="STRING",
                description="Order number formatted as ORD-XXXXX. Null if not provided.",
                nullable=True
            ),
            "intent": types.Schema(
                type="STRING",
                enum=["refund_request", "order_status", "billing_dispute", "other"],
                description="Primary intent classification."
            ),
            "urgency": types.Schema(
                type="STRING",
                enum=["low", "medium", "high", "critical"],
                description="Assessed customer urgency."
            ),
            "summary": types.Schema(
                type="STRING",
                description="A concise one-sentence factual summary of the issue."
            )
        },
        required=["intent", "urgency", "summary"]
    )
)

tools = [types.Tool(function_declarations=[extract_schema])]

# 2. Force the model to invoke this specific tool
tool_config = types.ToolConfig(
    function_calling_config=types.FunctionCallingConfig(
        mode="ANY",  # Must call a tool
        allowed_function_names=["extract_support_ticket"]
    )
)

Executing the Extraction

python

customer_text = "Hi, my name is Alex. I ordered the leather boots (ORD-98211) and they came in the wrong size. I need a replacement before my flight tomorrow morning!"

response = client.models.generate_content(
    model=model,
    contents=customer_text,
    config=types.GenerateContentConfig(
        tools=tools,
        tool_config=tool_config,
        temperature=0.0
    )
)

# Extract structured arguments directly (no json.loads required!)
function_call = response.candidates[0].content.parts[0].function_call
extracted_data = dict(function_call.args)

import json
print(json.dumps(extracted_data, indent=2))

# --- Execution Output ---
# {
#   "customer_name": "Alex",
#   "order_id": "ORD-98211",
#   "intent": "refund_request",
#   "urgency": "critical",
#   "summary": "Customer received wrong size leather boots (ORD-98211) and needs replacement before tomorrow morning flight."
# }

Result Payload

json

{
  "customer_name": "Alex",
  "order_id": "ORD-98211",
  "intent": "refund_request",
  "urgency": "high",
  "summary": "Customer received incorrect size for leather boots under order ORD-98211 and urgently needs a replacement before traveling tomorrow."
}

Comparing Extraction Techniques

Dimension	Plain Text Prompting	Forced Function Calling
JSON Parse Reliability	~80–90%	100% Guaranteed
Enum Enforcement	Model might invent new enums	Strictly bound to enum array
Type Safety	Strings might be returned for ints	Adheres to declared JSON Schema types
Post-Processing Overhead	High (regex, strip markdown)	Zero (direct Python dict)

Key Takeaways

Never Prompt for JSON in Critical Workflows: Use forced function declarations (mode="ANY") for enterprise data extraction.
Explicit Enums & Nullability: Clearly declare enum lists and mark optional fields with nullable=True.
Deterministic Parsing: Eliminates brittle regex wrappers and JSONDecodeError exceptions in production.

What's Next

In Part 8: Building a Production Extraction Pipeline with Confidence Scoring, we build an intelligent routing pipeline with self-assessed confidence scores and automated human review escalation.