Bonus: Tokenization, Cost Strategy & Total Cost of Ownership for ADK Agents

📦 Companion Code: View the complete, runnable code on GitHub: part16 in the cartpilot-adk2-series repository.

Part 10 measured what CartPilot's conversations cost using after_model_callback. That answers "what did this call actually cost." This part answers two questions Part 10 didn't: what determines that cost before you ever make the call, and what does CartPilot cost as a whole system, once you count everything beyond raw LLM tokens — hosting, third-party tools, CI, human review, logging. Token cost is one line item in a much longer bill.

Tokenization, briefly

A token isn't a word — Gemini models tokenize at roughly 4 characters per token, so "CartPilot" is 2-3 tokens, not one, and "the" might be a whole token or share one with surrounding punctuation depending on context. You don't need to predict exact tokenization by hand; you need a way to measure it before you spend money on a real call.

Counting tokens before you spend anything

The google-genai SDK gives you two ways to count tokens pre-flight — no generation, no output cost:

# token_budget.py
from google import genai

MODEL = "gemini-3.5-flash"

SEARCH_AGENT_INSTRUCTION = """
You are the CartPilot product search assistant.
- Use `search_products` to find items matching what the shopper describes.
- Use `get_product_details` when the shopper wants more info on one item.
- If `search_products` returns no results, say so and suggest broadening the search.
- Don't call a tool more than once for the same query.
- Keep responses short (2-4 sentences).
"""


def count_offline(text: str) -> int:
    """Zero-cost, zero-network token count — safe to call thousands of
    times (e.g. in a CI check that fails if an instruction grows past
    a budget) without touching the API at all."""
    tokenizer = genai.LocalTokenizer(model_name=MODEL)
    return tokenizer.count_tokens(text).total_tokens


def count_online(client: genai.Client, text: str) -> int:
    """A real API call — use for an exact figure on a specific request
    shape the local tokenizer can't approximate (tool schemas, multimodal)."""
    return client.models.count_tokens(model=MODEL, contents=text).total_tokens


if __name__ == "__main__":
    import os
    from dotenv import load_dotenv

    load_dotenv()

    # 1. Offline count — zero cost, zero network, runs in sub-millisecond.
    offline_tokens = count_offline(SEARCH_AGENT_INSTRUCTION)
    print(f"Offline token count: {offline_tokens} tokens")

    # 2. Online count — validates the local tokenizer against the real API.
    api_key = os.getenv("GEMINI_API_KEY")
    if api_key:
        client = genai.Client(api_key=api_key)
        online_tokens = count_online(client, SEARCH_AGENT_INSTRUCTION)
        print(f"Online API count:    {online_tokens} tokens")
        print(f"Difference:          {abs(online_tokens - offline_tokens)} tokens")
    else:
        print("Set GEMINI_API_KEY in .env to run the online count comparison.")

pip install "google-genai[local-tokenizer]"
python token_budget.py

Two different tools, two different jobs:

• LocalTokenizer — fully offline, free, unlimited calls. Use it in CI: fail a build if an agent's instruction grows past a budget you've set, the same way you'd fail a build on a bundle-size regression.
• count_tokens() (a real, essentially-free API call) — use when you need an exact figure that accounts for things the offline tokenizer can't see cleanly, like tool schemas or multimodal content.

The point of measuring an instruction's token count in isolation: that number is a permanent per-call tax. Every single one of search_agent's calls re-sends its full instruction — a system instruction that grows from 60 tokens to 200 tokens isn't a one-time cost, it's a 3x-plus multiplier on that specific line item, forever, on every call that agent ever makes. This is the same discipline Part 10 introduced ("trim instructions like a hot code path") — now with a tool to actually check it instead of eyeballing it.

The cost strategy checklist

Nothing here is new — it's every cost lever this series introduced, in one place:

Every one of these is a lever on the LLM API cost line specifically. TCO needs more than that line.

Total Cost of Ownership: the bill beyond tokens

A CartPilot-in-production bill has at least six line items, and LLM tokens are usually not the largest one at moderate scale:

1. LLM API cost (Part 10) — what we've been measuring all along.
2. Compute/hosting (Part 14) — Cloud Run or Agent Runtime, billed independently of token usage.
3. Third-party MCP tools (Part 13) — a shipping-rate provider, a tax API, anything you don't control the pricing of.
4. Eval/CI runs (Part 12) — every eval suite run against a real model costs real tokens, and it runs on every deploy, not just once.
5. Human-in-the-loop time (Part 8) — a refund approval isn't free; someone's minutes have a cost.
6. Observability storage (Part 11) — structured audit logs at scale aren't free to store or query.

# tco_calculator.py
from dataclasses import dataclass

@dataclass
class AgentUsage:
    name: str
    calls_per_conversation: float
    avg_input_tokens_per_call: int
    avg_output_tokens_per_call: int
    price_per_million_input: float
    price_per_million_output: float

    def cost_per_conversation(self) -> float:
        input_cost = (self.avg_input_tokens_per_call * self.calls_per_conversation
                       / 1_000_000 * self.price_per_million_input)
        output_cost = (self.avg_output_tokens_per_call * self.calls_per_conversation
                        / 1_000_000 * self.price_per_million_output)
        return input_cost + output_cost


@dataclass
class TCOAssumptions:
    conversations_per_month: int
    agents: list[AgentUsage]
    hosting_monthly_usd: float = 150.0
    mcp_calls_per_conversation: float = 0.3
    mcp_cost_per_call_usd: float = 0.01
    deploys_per_month: int = 20
    eval_cases: int = 25
    avg_eval_cost_per_case_usd: float = 0.002
    refunds_needing_approval_per_month: int = 40
    avg_approval_review_minutes: float = 3.0
    reviewer_hourly_rate_usd: float = 35.0
    log_events_per_conversation: float = 12.0
    log_storage_cost_per_million_events_usd: float = 5.0


def compute_tco(a: TCOAssumptions) -> dict:
    llm_cost_monthly = sum(agent.cost_per_conversation() for agent in a.agents) * a.conversations_per_month
    mcp_cost_monthly = a.mcp_calls_per_conversation * a.conversations_per_month * a.mcp_cost_per_call_usd
    eval_cost_monthly = a.deploys_per_month * a.eval_cases * a.avg_eval_cost_per_case_usd
    hitl_hours = a.refunds_needing_approval_per_month * a.avg_approval_review_minutes / 60
    hitl_cost_monthly = hitl_hours * a.reviewer_hourly_rate_usd
    log_cost_monthly = (a.log_events_per_conversation * a.conversations_per_month
                         / 1_000_000 * a.log_storage_cost_per_million_events_usd)
    total = (llm_cost_monthly + a.hosting_monthly_usd + mcp_cost_monthly
             + eval_cost_monthly + hitl_cost_monthly + log_cost_monthly)
    return {
        "llm_api_cost": round(llm_cost_monthly, 2),
        "hosting_cost": round(a.hosting_monthly_usd, 2),
        "mcp_tool_cost": round(mcp_cost_monthly, 2),
        "eval_ci_cost": round(eval_cost_monthly, 2),
        "human_review_cost": round(hitl_cost_monthly, 2),
        "observability_cost": round(log_cost_monthly, 2),
        "total_monthly_usd": round(total, 2),
        "cost_per_conversation_usd": round(total / a.conversations_per_month, 4),
    }

Run it with CartPilot's own architecture as the worked example (50,000 conversations/month, four agents from Parts 6-8, support_agent on the Pro tier):

python tco_calculator.py

=== CartPilot Monthly TCO ===

LLM cost by agent:
  coordinator          $     63.00
  search_agent         $     44.10
  cart_agent           $     47.85
  support_agent        $    126.00

LLM API (total)          $    280.95
Hosting / compute        $    150.00
Third-party MCP tools    $    150.00
Eval suite / CI          $      1.00
Human-in-the-loop time   $     70.00
Observability storage    $      3.00
--------------------------------------
TOTAL / month            $    654.95
Cost per conversation    $    0.0131

At this illustrative volume, LLM API cost is under half the total bill — hosting and third-party MCP tools together outweigh it. That's not a universal result (it flips at very high volume, where token cost dominates), but it's the actual point: guessing which line item matters most is exactly what this calculator replaces.

Using this for real decisions

Two things this model is actually good for:

• Justifying engineering investment. "Should we spend a sprint on context caching for cart_agent's tool schemas?" is answerable directly — change avg_input_tokens_per_call to reflect the post-caching estimate, rerun, and see whether the saved dollars justify the engineering time at your actual conversation volume. At 5,000 conversations/month the answer might be no; at 500,000 it's very likely yes.
• Feeding it real numbers, not guesses. Every field in TCOAssumptions maps to something you can pull from what this series already built: avg_input_tokens_per_call from Part 10's usage_log, refunds_needing_approval_per_month from Part 8's confirmation events, log_events_per_conversation from Part 11's audit trail. Once CartPilot has real production traffic, replace every illustrative number here with a measured one — the calculator becomes a live model of your actual bill, not a guess dressed up as one.

Where this leaves the series

Sixteen parts in, CartPilot has gone from a single agent with no tools to a fully specified production system — and now a way to reason about what running it actually costs, end to end, not just per model call. The habit worth keeping past this series: cost isn't a deployment afterthought bolted on once something works. Every part from 1 through 16 made a cost decision as part of the design itself — that's the difference between an agent system you can run confidently at scale and one that quietly becomes unaffordable the moment it succeeds.