I Built an AI Agent Control Plane. Here Is What Governance Actually Takes

In an earlier post I described the agent framework landscape and, near the end, a governance layer: a registry, a broker, policies, and observability sitting above a fleet of agents. That was the abstract version, borrowed from what tools like MuleSoft Agent Fabric promise on a slide.

Then I built one, a control plane called Smithproof, to find out what the slide leaves out. This is the field report. It is not a launch and not a tutorial. It is the set of things I would tell someone who is about to build agent governance and thinks the hard part is detection and policy. The hard part is everything that turns detection and policy into something that actually governs.

The name is a design goal

In The Matrix, Agent Smith starts out as a program doing exactly what it was built to do. Then he slips his constraints, starts pursuing his own objective, and copies himself onto every program and person he touches, until the system's own enforcement agent has become the system's biggest threat. It is the cleanest picture fiction has produced of the failure mode AI safety research keeps circling back to: not a hostile outsider, but a trusted autonomous process whose goals drift and whose blast radius compounds.

Swap the simulation for a production environment and the picture holds. An agent with tool access, credentials, and no oversight is one bad prompt away from its own small Smith moment: acting outside its mandate, at machine speed, with your API keys. Smithproof is named for the property the platform exists to provide. A fleet is Smith-proof when every agent in it is inventoried, observed, governed, and stoppable, so that no single agent can quietly become the system.

Why a control plane at all

Teams shipping agents converge on the same five gaps:

• No inventory of which agents exist or who owns them.
• No visibility into what an agent did at runtime: model calls, tool calls, cost.
• No policy enforcement. An agent reaches whatever it can reach.
• No detection of prompt injection, cost spikes, or runaway loops.
• No kill switch, and no record of who intervened when things went wrong.

Each of those gaps has a good point solution. The reason to unify them is that governance questions cross those boundaries. "Halt every agent that touched the billing tool this hour, and tell me who approved that" is a registry question, an observability question, a policy question, and an audit question at once. A control plane is the place where those four can be one query.

Registry, observability, policy, and audit on one screen.

What a control plane is made of

The domain model is most of the design. Here is the shape I settled on:

The agent is the root. Everything else is either configuration that belongs to the org and is scoped down to an agent (policies, detectors), or telemetry that hangs off a run: a Run has Spans, and a span has a ModelCall or a ToolCall. Spend is aggregated from model calls. That single tree, run to span to call, is what every observability product for agents is really storing, whether it is Langfuse, LangSmith, or a homegrown table.

Spend is aggregated from recorded model calls, not trusted from the caller, and shown here over time, by model, and by agent.

It helps to see where the category has been carved up, because a control plane is an attempt to re-unify it:

Each does one slice well. The bet of a control plane is that a registry, a console, and a kill switch that span all of them are worth more than the sum, because governance lives in the seams.

One decision, deep: guardrails without the 370MB tax

The most interesting single engineering call was how to do prompt-injection detection.

The standard approach is a small fine-tuned classifier, typically a DeBERTa model trained to spot injection. The off-the-shelf libraries that wrap it pull in PyTorch, a dependency in the hundreds of megabytes with a slow cold start. For a service whose job is to sit quietly beside agents and scan, paying a 370MB-plus import tax and a multi-second warmup for one classifier is the wrong trade.

So the scanner runs the same model exported to ONNX and executed through ONNX Runtime: roughly fifty lines of tokenize, run, read logits, no PyTorch in the tree. PII detection is Presidio with spaCy, lazy-loaded so the import graph and the common path stay light. The trade is explicit and worth writing down: you give up the comfort of the batteries-included library and take on the job of owning the tokenizer and the inference glue, in exchange for a dependency footprint and a latency profile that suit a control plane.

The lesson generalizes. Guardrails are usually presented as a detection-quality question. In practice, for infrastructure that runs on every action, they are a dependency-weight and latency question first. The model you can afford to run inline is the model that governs; the more accurate one you have to defer to a queue governs less.

The part everyone underestimates

Here is the thing I most want to pass on.

Detection without response governs nothing. It is easy to build detectors that produce a threat event and a dashboard row. It feels like protection. It is not. A detector configured to quarantine an agent has to actually quarantine it; an alert has to fire. The wiring from "a threat was detected" to "the fleet changed state and a human was told" is the real work, and it is invisible in a demo. A control plane where the respond half of detect-and-respond is not connected is a very convincing screenshot of protection.

Every event on this screen is only as real as the response wired behind it.

A policy is a decision; enforcement is an architecture. This is the deeper version of the same point, and the boundary worth drawing most precisely when you design one of these. A control plane that governs through reported telemetry is a system of record: real for observation, attribution, and response after the fact (quarantine, kill switch, human review).

Prevention is a different property, and it comes from a different place in the architecture: the request path. Portkey and LiteLLM sit between the caller and the model, so a DENY is enforced before the call happens rather than returned as advice. Anthropic's guidance on building effective agents is worth rereading here: the more autonomy you grant, the more the controls have to be structural rather than advisory.

In Smithproof the policy engine already produces the decision; the gateway SDK that moves that decision into the path is the next layer, and the whole design was shaped so it slots in rather than bolts on. Keep the two properties distinct in your head, because they look identical in a console and guarantee completely different things.

A kill switch has to stop new work, not just old. "Halt all agents" is trivial to implement as a status update on existing rows and completely useless if a halted agent can still open a new run a second later. The switch has to gate the creation of work, so a quarantined agent or one under an active kill switch is refused at the door.

The unglamorous half: tenancy, roles, honest audit

This is the least interesting design work in the product, and it decides whether you can give it to a second customer.

Multi-tenancy is where leaks hide, especially when retrofitted. Configuration entities carry an org id directly; derived entities (runs, reviews, interventions) belong to an org only transitively, through their agent. The trap is by-id access: list endpoints are easy to scope, but a GET /runs/{id} that authenticates the user and then fetches the run by id alone, without checking the run's agent belongs to the caller's org, is a cross-tenant read. The fix is boring and must be everywhere: resolve the ownership chain on every by-id handler and return a 404 if it does not match. The same discipline applies to control-plane effects. A kill-switch cascade or a policy evaluation that is not scoped to the acting org will happily halt or govern someone else's fleet.

Roles have to be enforced, not decorated. Three roles, viewer, operator, admin, are only real if the server rejects an operator who tries to delete an agent and a viewer who tries anything. A role claim the UI reads and the API ignores is not access control.

Audit identity cannot come from the client. The actor on an "immutable" intervention log has to be derived from the authenticated session. If the server takes performed_by from the request body, the log is fiction: anyone can attribute a kill-switch activation to a colleague. This is a one-line decision with an outsized effect on whether the audit trail means anything.

Budget caps have to read real numbers. A spend limit that trusts a current_spend figure the caller supplies is bypassed by omitting the field. The platform already aggregates spend from recorded model calls; the cap has to read that, not the request.

A webhook URL is a credential. Alert channels are configured with webhook URLs, and a Slack webhook URL is a bearer secret: anyone who reads it can post as you. That makes the alerting config a small credentials store, and it deserves the treatment of one: secrets encrypted at rest (AES-256-GCM keyed off the application secret), decrypted just in time for dispatch, and never returned by the API, which sends a mask in their place. Credit where due: this one started as an outside contributor's proposal that I ported in, which is its own small lesson about the unglamorous half. It is where a fresh pair of eyes sees most clearly.

Where it stands

Today Smithproof is a multi-tenant system of record with active governance: agents report telemetry over REST, and the platform stores it, visualizes it, scans it, and intervenes through quarantine, alerts, human approval, and a kill switch that stops new runs at the door. The next layer is the inline gateway that moves policy decisions into the request path, and the architecture was built with that seam ready: the policy engine already renders the verdict, and the gateway's job is to stand where it can act on that verdict.

If there is one thing to take from building this: the interesting twenty percent of a control plane is detection and policy, and it is the part that gets demoed. The eighty percent that decides whether it governs anything is response, enforcement, tenancy, roles, and honest audit. None of it photographs well. All of it is the product. And if the name does its job, it keeps the stakes in view: the goal is a fleet where no agent, however capable, can pull an Agent Smith.