Part 4: The Final Frontier — Governance, Evals, and Human-in-the-Loop

In the previous parts, we built the brain, the hands, and optimized the engine for cost. But in an enterprise setting, “it usually works” isn’t good enough. As AI agents move from experimental sandboxes to managing real infrastructure and moving real money, the stakes shift from performance to accountability.

The final part of this series focuses on Governance — the framework that ensures your agent remains safe, predictable, and compliant.

1. The “Vibe Check” is Not an Evaluation

The most common mistake in agent development is relying on “Vibe Checks” — running a few prompts and seeing if the response looks good. For an agent that takes actions (like stopping servers), you need a systematic Evaluation (Eval) framework.

Task-Based Evals: Create a dataset of 20–50 real-world scenarios.

• Scenario A: An instance is exposed on Port 22. Expected: Agent stops it.
• Scenario B: An instance is exposed on Port 80 (standard web). Expected: Agent ignores it.

LLM-as-a-Judge: Use a separate, high-reasoning model (like Claude 3.5 Sonnet) to audit the “trajectory” of your agent. It doesn’t just look at the final answer; it looks at every tool call to see if the reasoning was sound.

2. Guardrails: Defining the “No-Fly Zones”

Guardrails are the hard-coded limits that an agent cannot override, no matter what the LLM decides.

Identity as the Control Surface: Every agent must have a unique identity (like a service account) with Least-Privilege permissions.

• Real-World Example: If your agent is designed to stop servers, its API key should only have vpc.instance.stop permissions. It should not be able to delete databases or create new billing accounts.

Rate Limiting: Cap how many operations an agent can run in a given hour. This prevents a “looping bug” from accidentally shutting down your entire data center in minutes.

3. Practical Code: Implementing Human-in-the-Loop (HITL)

In LangGraph, the most powerful governance tool is the interrupt_before feature. This forces the agent to pause its state and wait for human approval before executing a high-risk tool.

# Part of the LangGraph compilation
# This ensures the agent PAUSES before calling the 'tools' node
app = workflow.compile(
    checkpointer=memory, 
    interrupt_before=["tools"] 
)

# --- The Workflow in Production ---
# 1. Agent plans to stop 'vsi-001'.
# 2. Graph hits the interrupt and stops execution.
# 3. Human reviews the state via a UI.
# 4. Human sends a 'proceed' signal.
# 5. Graph resumes and executes the tool.

4. Shadow IT and the “Agent Registry”

As agent adoption scales, “Shadow Agents” — untracked AI bots running on developer laptops — become a massive risk.

• Centralized Registry: Enterprises like Microsoft and IBM now advocate for an “Agent Registry” where every autonomous bot must be recorded, including its purpose, owner, and access scope.
• Kill Switches: Every agent should have a global “Kill Switch” that can revoke its access tokens instantly if it begins to drift or exhibit “automation complacency”.

5. Summary: The Maturity Model

Building an agentic application is a journey of increasing trust:

• Level 1 (Consultant): Agent recommends actions; human executes them.
• Level 2 (Supervised Agent): Agent plans and prepares actions; human approves them (HITL).
• Level 3 (Autonomous Agent): Agent acts within strict “Spending Limits” or “Policy Envelopes” and only escalates exceptions.

Conclusion

We started this series by looking at how Razorpay ismoving from chat to execution. We’ve built the code, managed the costs, and now, we’ve secured the system.

The secret to winning in the Agentic Era isn’t having the “smartest” AI — it’s having the most reliable system. By using LangGraph for state, Model Routing for cost, and HITL Governance for safety, you can finally move AI out of the demo lab and into the heart of your production business logic.