From Demo to Production: The Guardrails That Make an AI Agent Safe to Ship
Hook: Most "AI agents" you see on the internet are demos. Here's the single most common
reason they never reach production — and a small, open-source harness that gets past it.
We are past the phase where the hard part of building an AI agent was calling the model.
The hard part now is the 10% nobody talks about: what stops the agent from doing something
harmful? I've seen this from both sides — I built and ran a ~25-agent platform in production
at Microsoft, and now I help teams take agent ideas from a notebook to real users.
The uncomfortable truth: a chatbox that can call 5 tools is not a product. The difference
between a weekend project and a system you can put in front of customers is three things —
and they're all boring, non-glamorous engineering:
- How you grade output quality (the quality gate).
- How you decide when a human must sign off (the approval gate).
- How you make the whole thing model-agnostic so you're not locked into one vendor.
So I wrote a tiny harness that keeps these front and center. It's intentionally small — small
enough to read in an hour — because the value isn't in a framework, it's in the pattern.
1. The quality gate: don't ship what you can't grade
An agent's output is a prediction, not a promise. Before it ships, you need a check that
it passes your bar. In the harness this is a pluggable QualityGate — a rule of thumb you swap
with an LLM judge or a test suite:
# agent_harness/eval.py
@dataclass
class EvalReport:
passed: bool
score: float
checks: List[str]
class QualityGate:
def grade(self, proposal: str, context: str = "") -> EvalReport:
return self.grader(proposal, context)
The loop refuses to execute if the gate fails:
result.report = self.quality.grade(proposal, f"state={state}")
if not result.report.passed:
self.approval.log("quality-gate", "blocked", result.report.__str__())
return result
Notice it logs the block. In production you'd want every blocked attempt in your
observability stack. "We rejected 12% of agent proposals this week" is a real KPI — it means
the gate is doing its job.
2. The approval gate: the step everyone forgets
This is the one that actually gets enterprises to say yes. When an agent wants to expedite
an order, cancel a subscription, or move money, it should stop and ask a human. Silence is
not consent.
# agent_harness/approval.py
class ApprovalGate:
def request(self, action: str, detail: str) -> bool:
# In production: push a notification to Teams / Slack / email and wait.
decision = input(f"Approve {action}? [y/N] ").strip().lower()
self.audit.append(AuditEntry(time.time(), action, "human-reviewer", decision, detail))
return decision.startswith("y")
In the scaffold, marking a tool needs_approval=True is enough to route it through the gate:
@tool("expedite_order", "Mark an order as expedited.", needs_approval=True)
def expedite_order(order_id: str) -> str:
return f"PO {order_id}: marked expedited"
And because there's an audit trail, you can always answer "who changed this and why?" —
which is usually the first question a compliance team asks.
3. Model-agnostic providers: don't marry one vendor
Models change every few weeks, and so do prices. Your agent loop should never know which
vendor it's talking to:
# agent_harness/providers.py
class ModelProvider(Protocol):
def complete(self, messages: List[Message], tools: Dict[str, Any]) -> str: ...
agent = Agent(provider=default_provider("openai")) # or "deepseek", "qwen", "mock"
A few lines of abstraction mean you can run the same agent on OpenAI, Azure OpenAI, DeepSeek,
or Qwen with a config change — and test the whole thing offline with a MockProvider that
needs no API key. That's not just engineering hygiene, it's a cost lever and a hedge.
4. Business workflows are state machines, not a chat loop
The other huge pattern: don't let an "agent" freewheel over your critical process. Model the
workflow explicitly. Here's a purchase-order exception state machine — the kind of thing that
shows up in every supply chain / ERP copilot:
# agent_harness/state_machine.py
class State(str, Enum):
OPEN = "open"; EXPEDITED = "expedited"; CANCELLED = "cancelled"
EXECUTED = "executed"; CLOSED = "closed"
class POStateMachine:
def transition(self, target: State, reason: str = "") -> State:
if target not in TRANSITIONS[self.state]:
raise InvalidTransition(f"{self.state} -> {target}")
self.state = target
return self.state
open ──► expedited ──► executed ──► closed
│ ▲ ▲
└──exception──┘ (human approval gate in between)
The agent proposes a transition; the state machine enforces what's legal; a human approves
anything consequential. That's how you get autonomy with control — the exact phrase
enterprises want to hear.
Put it together
agent = Agent(provider=provider, registry=registry,
approval=ApprovalGate(), quality=QualityGate())
sm = POStateMachine(po_id="PO-1234")
result = agent.run("PO-1234 is stuck; expedite it.", state=sm)
print(result.audit)
Pull the repo and run it with no API key:
python examples/po_workflow.py --yes
You'll see the quality gate pass, the human approve, the order get expedited, the state move to
expedited, and the whole thing recorded in the audit trail. That's a real production-shaped
flow in ~300 lines.
The takeaway
If you're building an agent, build the guardrails first. It's the difference between a
demo you screenshot and a system your business actually trusts. And if you don't want to build
it yourself — or want someone who's shipped this at Microsoft scale — get in touch.
Open-sourced under MIT. Fork it, build on it, and say hi.
About the author: I'm Zhaowei Sun, an AI agent / Copilot engineer who's built and run
production agent platforms at Microsoft and high-scale systems at Hulu. I consult on agent
architecture, and I keep this repo as the scaffolding I wish I'd had.
Tags: AI Agents, LLM, RAG, Microsoft Copilot, Software Architecture, System Design
Suggested platforms: Dev.to | Medium | LinkedIn | Hacker News (Show HN) | 掘金 (translated)
This article was originally published by DEV Community and written by sunny 1024k.
Read original article on DEV Community