LangGraph Series, Part 6: The Business Playbook I — Operations and Cost

Part 6 of 7. Parts 1–5 were about building the thing. Parts 6–7 are about deciding whether, and how, to actually run it as a business — starting with the two questions that come up in nearly every budget and roadmap conversation: operations and cost.

This post is deliberately not framework-neutral fluff — every recommendation below ties back to something concrete in the graph structure from Part 5, so you can see exactly where each cost or operational concern actually originates in the code, not just as an abstract "AI is risky" warning.

Operations

Why observability isn't optional here, specifically

In a single prompt-call system, there's exactly one thing to observe: did the output look right. In the Part 5 graph, a single customer message can take multiple different paths (three intents, two of which lead to different endings), and the actual path taken for any given request is decided at runtime, not fixed in advance. If a customer complains "the bot gave me a weird answer," your first question has to be "which path did it take, and why" — and without tracing, the honest answer is "we don't know," because the routing decision (classify_intent's output) isn't visible unless you specifically capture and expose it.

What this means practically: use a tracer (LangSmith or an equivalent) from the first production deployment, not after the first incident. What you specifically want visible per run: which node ran, in what order; what classification_reasoning said; what the draft was before human review; and what the human decided. All five of these map directly to fields already in the Part 5 state — the tracing requirement isn't extra engineering, it's exposing state you're already computing.

Why the deployment model decision (self-host vs. managed) should be made before writing more code

Self-hosting (a Docker container running the compiled graph behind your own API) gives full control over infra, cost, and data residency, at the cost of building and maintaining: the API layer, the human-review inbox UI, horizontal scaling for concurrent conversations, and the checkpoint database's ops (backups, connection pooling, monitoring). A managed runtime (e.g., LangGraph Platform) provides most of this out of the box, including a built-in human-in-the-loop review interface, at the cost of less infrastructure control and a per-usage or platform fee.

The decision rule: if your team already operates production databases and API services comfortably, and data residency or cost-at-scale considerations favor owning the infrastructure, self-host. If the team's strength is in the agent logic itself and not in operating stateful infrastructure, the managed option's built-in approval-queue UI and scaling alone often pay for themselves in the engineering time not spent building an internal admin tool from scratch — building a decent human-review dashboard is a bigger project than it sounds.

Why paused runs are an operational category of their own

A request sitting in human_review, waiting on a manager, isn't "finished" and isn't "failed" — it's a third state most standard application monitoring doesn't have a name for. If nobody is watching for it, a paused refund approval can sit for days, and from the customer's perspective, the business simply never responded.

What this requires operationally: an explicit alert on interrupts that have been pending past a defined SLA (e.g., "no approval decision within 4 business hours"), routed to whoever owns that queue — not just relying on someone happening to check a dashboard. This is a genuinely new kind of alert most teams don't have a template for yet, and it's worth setting up before launch, not after the first customer complaint about a "ghosted" refund.

Why a hard iteration cap is an operational safeguard, not just a cost one

Part 3 covered why unbounded tool-calling loops are a cost risk. They're also an operational one: a stuck loop consuming API calls at machine speed can, in a bad enough case, hit rate limits that then degrade every other concurrent conversation hitting the same API key. Treat "a single run has been looping past N iterations" as its own alertable incident class, separate from generic error monitoring — it has a distinct root cause (agent confusion, not infrastructure failure) and a distinct fix (fix the prompt or add a stricter exit condition, not restart a server).

Cost

Why cost modeling has to happen per-node, not per-request

The Part 5 graph makes two LLM calls minimum for every request (classify, then draft) and can make more once tools or retry loops are added. A cost estimate based on "roughly one API call per customer message" will be wrong by 2–5x the moment real usage includes tool-calling loops, retries, or multi-agent handoffs. The right cost model is:

cost per resolved request = Σ (LLM calls per node × tokens per call × that node's model price) + human-reviewer time for flagged requests

This is worth building as an actual spreadsheet before launch, node by node, because it's the only way to answer the question stakeholders will actually ask: "what does this cost per 1,000 support tickets," and it's the only way to spot the specific node driving cost if that number comes in higher than expected later.

Why classification uses a cheaper model than drafting, as a cost decision explicitly

This was mentioned in Part 5 as a design choice; here's the cost reasoning in full. Classification is a bounded, low-ambiguity task (one of three categories) — a smaller, cheaper, faster model handles this reliably. Drafting a customer-facing response is open-ended generation where a stronger model's better judgment and tone genuinely show up in output quality. Running both stages on the strongest available model spends the same premium price on the easy task as the hard one, for no quality benefit on the easy task. In a graph with several nodes, auditing "does this specific node actually need the frontier model, or would a smaller one do the same job for a fraction of the cost" is routinely a 40–70% cost reduction with no measurable quality loss on the nodes that get downgraded — but it has to be checked node by node, not assumed for the whole graph at once.

Why caching belongs in the tool layer, not just at the LLM call layer

If a conversation calls get_order_status("A123") twice within the same session (a common pattern — the LLM re-checks context after a tool loop), that's a wasted, identical database round-trip and, more importantly, wasted tokens re-processing the same result in the next LLM call. Caching tool results within a conversation's state (or in a short-lived cache keyed by tool name + arguments) avoids paying for the same lookup twice within one run. This is a small implementation detail with a real effect at volume — the savings scale with how often your specific agent's loops re-derive the same fact.

Why human-review time is a cost line item, not a footnote

The Part 5 design routes every refund through a human. At launch, expect approval rates to be high (the drafts are usually reasonable) — which means most of that reviewer time is spent confirming something that was already correct. This is not wasted effort (it's the safety mechanism doing its job), but it is a real, budgetable labor cost that scales with refund volume, and it should be sized and staffed before launch, not discovered as a bottleneck once refund requests start piling up in a queue nobody budgeted reviewer time for. As trust in the system builds and you have data on approval accuracy, this is also the metric that justifies (or doesn't) narrowing the approval requirement — e.g., only routing refunds above a dollar threshold to a human, once you have evidence low-dollar refunds are reliably correct.

The cost trade-off with hallucination mitigation, named honestly

Every mitigation in Part 7 — human review, verification nodes, retrieval grounding — costs something: reviewer time, an extra LLM call, or engineering effort to wire up a data source. There is no zero-cost way to reduce hallucination risk to zero. The realistic framing for a business conversation is: spend mitigation budget proportionally to the cost of being wrong — heavy mitigation (human review) on refunds and anything touching money or irreversible actions, lighter mitigation (a cheaper verification pass, or none) on a general FAQ answer where being slightly off costs a follow-up message, not a financial loss. Applying the same heavy mitigation everywhere is both unaffordable and unnecessary; applying none anywhere is the incident that ends up on your postmortem doc.

What's next

Part 7 covers the other half of the business conversation: maintainability as the system and team grow, concrete hallucination mitigations, security and governance concerns specific to agentic systems, and closes the series with a decision framework for when LangGraph is — and isn't — the right tool at all.