Token Economics & Cost Optimization for Agents

📦 Companion Code: View the complete, runnable code on GitHub: part10 in the cartpilot-adk2-series repository.

Every part so far has mentioned cost in passing — model tiering in Part 1, tool design in Part 2, token growth in Part 3, structured output in Part 4, deterministic graph steps in Part 5, single-turn tasks in Part 7, bounded retries in Part 9. Today we stop mentioning it and start measuring it, because every one of those earlier decisions was a guess until you can see the actual numbers behind it.

The hook: after_model_callback

ADK fires after_model_callback after every single model call, handing you the raw LlmResponse — including usage_metadata: prompt tokens, output ("candidate") tokens, total tokens, and (when context caching is active) how many of those input tokens were served from cache rather than billed fresh.

# cartpilot/cost_tracking.py
from google.adk.agents.callback_context import CallbackContext
from google.adk.models.llm_response import LlmResponse

# Illustrative only — check https://ai.google.dev/gemini-api/docs/pricing
# for current rates before using this for real budgets.
PRICE_PER_MILLION_TOKENS = {
    "gemini-3.5-flash": {"input": 0.30, "output": 2.50},
    "gemini-2.5-pro": {"input": 1.25, "output": 10.00},
}

USAGE_LOG_KEY = "usage_log"


def track_token_usage(callback_context: CallbackContext, llm_response: LlmResponse) -> None:
    """after_model_callback: records token usage and estimated cost for
    every model call, tagged by which agent made it."""
    if not llm_response.usage_metadata:
        return

    usage = llm_response.usage_metadata
    prices = PRICE_PER_MILLION_TOKENS.get("gemini-3.5-flash")

    input_tokens = usage.prompt_token_count or 0
    output_tokens = usage.candidates_token_count or 0
    cached_tokens = getattr(usage, "cached_content_token_count", 0) or 0

    billable_input_tokens = max(input_tokens - cached_tokens, 0)
    estimated_cost = (
        billable_input_tokens / 1_000_000 * prices["input"]
        + output_tokens / 1_000_000 * prices["output"]
    )

    entry = {
        "agent": callback_context.agent_name,
        "input_tokens": input_tokens,
        "cached_tokens": cached_tokens,
        "output_tokens": output_tokens,
        "estimated_cost_usd": round(estimated_cost, 6),
    }
    log = callback_context.state.get(USAGE_LOG_KEY, [])
    log.append(entry)
    callback_context.state[USAGE_LOG_KEY] = log


def summarize_usage(state: dict) -> dict:
    """Rolls up the per-call log into per-agent and total cost."""
    log = state.get(USAGE_LOG_KEY, [])
    by_agent: dict[str, dict] = {}
    total_cost = 0.0
    for entry in log:
        agent = entry["agent"]
        bucket = by_agent.setdefault(agent, {"calls": 0, "input_tokens": 0,
                                              "output_tokens": 0, "cost_usd": 0.0})
        bucket["calls"] += 1
        bucket["input_tokens"] += entry["input_tokens"]
        bucket["output_tokens"] += entry["output_tokens"]
        bucket["cost_usd"] += entry["estimated_cost_usd"]
        total_cost += entry["estimated_cost_usd"]
    return {"by_agent": by_agent, "total_calls": len(log), "total_cost_usd": round(total_cost, 6)}

Wire it onto every agent in the tree:

# cartpilot/agent.py (excerpt)
from .cost_tracking import track_token_usage

for _agent in (search_agent, cart_agent, support_agent):
    _agent.after_model_callback = track_token_usage

root_agent = Agent(
    # ...
    after_model_callback=track_token_usage,
)

Every conversation now accumulates a usage_log in session state. Call summarize_usage(session.state) at the end of a run (or periodically in production) and you get a real answer to "which specialist is actually expensive?" — not a guess.

Reading the numbers, not just collecting them

Once you have per-agent breakdowns across a batch of real conversations, three patterns tend to show up:

1. The coordinator is often the highest-call-count agent, not the highest-cost one. It runs on every single turn (routing), but each call is short. A specialist that runs less often but generates long structured responses can cost more per conversation despite fewer calls. Don't assume "most frequent" means "most expensive" — check both.
2. Input tokens dominate in multi-agent systems, not output. Every specialist's system instruction, tool schemas, and any conversational history it inherits get re-sent as input on every call. A bloated instruction block is a recurring tax, paid every single turn — trim instructions the same way you'd optimize a hot code path.
3. Context caching pays off exactly where you'd expect: large, static, repeated content. If cart_agent's tool schemas and instruction are identical across thousands of conversations, that block is a candidate for caching — cached_content_token_count in the usage data tells you how much of your input is already benefiting versus how much is being paid for fresh every time.

Turning measurement into a tiering decision

This is where Part 1's "Flash vs Pro" question gets answered with data instead of intuition. A workable process, once you have a few hundred real conversations logged:

• Group usage_log entries by agent and look at cost per conversation, not per call.
• For any agent where cost is high and you have evidence of quality problems (from Part 12's eval suite) at the cheaper tier, that's your Pro candidate — not "support feels important so it should get the strong model."
• For agents with high call counts and low per-call complexity (routing, simple lookups), confirm they're on the cheapest viable tier and stay there.

Concretely for CartPilot: search_agent and cart_agent handle bounded, well-specified tasks — the eval suite in Part 12 should confirm Flash performs well on them, keeping them cheap permanently. support_agent is the one place worth watching — if evals show refund-policy edge cases or emotionally charged conversations produce worse outcomes on Flash, that's the specific, evidence-backed case for routing those conversations (not all of support) to Pro. That's a meaningfully different decision than "support agents should be smart," and it's one you can defend with numbers.

What's next

Cost tracking is one instance of a broader pattern: hooking into the agent lifecycle to observe and control behavior without changing what each agent does internally. Part 11 generalizes this with the full callback system — guardrails, structured logging, and policy checks alongside the cost tracking we just built.