Field notes from two years building production agents at a Fortune 100
Most writing about AI guardrails reads like a vendor pitch — a layered architecture diagram, a list of capabilities, a security-checklist deliverable. The reality of what actually keeps an enterprise AI agent system safe in production is narrower than that, less glamorous, and mostly stuff that existed before LLMs. IAM, network egress, audit trails, secrets management. With a few new controls bolted on for the agent layer.
I've spent the last two years building production AI Platform Agents at a Fortune 100 manufacturer — agents that diagnose CI/CD failures, triage Kubernetes incidents from a Microsoft Teams channel, generate infrastructure documentation from live repos and Terraform state, and offer real-time guidance on engineering pipelines. They run on EKS, talk to OpenAI APIs through a LangChain orchestration layer, and ground their answers in a RAG corpus we maintain.
What follows isn't a survey of every guardrail option. It's the layered stack we actually run in front of every agent, ranked by what would break first if you pulled it out. Plus a section on what's theater, and a section on what I got wrong.
The layered guardrail stack
Top-to-bottom. Each layer prevents a specific class of failure. The order matters — the earlier the layer fails, the more catastrophic the failure mode.
1. Identity at the agent boundary. Every agent runs as a workload identity (IRSA on EKS, equivalent on other clouds), never with shared service credentials. The agent's IAM scope is the security ceiling for everything it can do — no prompt, no model upgrade, no clever tool sequence can route around it. If an agent doesn't need read access to a database, the IAM role doesn't have it. This is the most boring and most load-bearing control we run. Nothing else on this list matters if the agent's IAM scope is too wide.
2. Tool allow-lists per agent. The model decides which tool to call. The platform decides which tools exist in that agent's environment. A code-search agent doesn't have a "send email" tool registered. A deploy-status agent doesn't have a "modify deployment" tool. The allow-list is a static config that ships with the agent, reviewed in code review like any other production config. Dynamic tool registration based on model output is a vulnerability class we explicitly don't enable.
3. Network egress controls. Agents reach allowlisted endpoints only. Outbound DNS filtering plus an egress proxy. This catches the failure mode where a model suggests a URL that shouldn't exist — typos, hallucinations, or rare cases of actual attack. We've never had an attack catch on this layer; we've had model hallucinations catch on it weekly.
4. Secrets isolation. Agents never see raw secrets. Vault and AWS Secrets Manager serve short-TTL session tokens injected at tool-call time, not into prompt context. The reasoning is simple: anything in prompt context is loggable, replay-able, and inspectable. Secrets aren't.
5. Audit trail for every model call and every tool call. Input, output, tool args, user identity, model version, request ID. Tamper-evident storage. This is the layer that makes incidents postmortem-able — when something goes wrong (and it does), you can reconstruct what the agent saw and what it decided.
6. Human-approval interrupts on irreversible actions. For tool calls that write to systems-of-record, the platform pauses and asks. The model never crosses that line on its own. The right phrase is no irreversible action without a human in the loop — every other control on this list is preventive; this one is the safety net.
What's NOT load-bearing
Most "AI guardrail" products optimize for the parts that don't matter. A few things I'd call theater after two years in production:
Prompt-level instructions to "never do X." Useful for behavior shaping. Useless as a security control. The model can be talked out of any instruction with enough indirection; the control has to be at the tool or IAM layer.
Generic PII detection filters. High false-positive rate, low actual value compared to scoping what data the agent could access in the first place. If the agent never saw the SSN because IAM said no, you don't need a PII filter on the output.
LLM-based "guardrail models" that grade the main model's output. Adds latency. Fails closed in ways you can't always afford. Doesn't reliably catch the failure modes you actually have in enterprise contexts. A second model voting on the first model's output is not the same thing as a security control; it's a model ensemble with extra steps.
Detailed "AI risk taxonomies" without the underlying controls. Mapping every possible failure mode to a row in a spreadsheet feels like governance. It isn't. If the spreadsheet doesn't tie back to a control in the layered stack above, it's an audit deliverable, not a control.
These tools help if you already have the layered stack. They aren't substitutes for it.
Governance beyond the model
Guardrails are runtime controls. Governance is the organizational frame that makes guardrails sustainable. The two are different problems with different owners, and conflating them is the most common enterprise mistake I've seen.
A few things that actually constitute governance, in our experience:
Per-agent ownership. Every agent has a named owner team. Tool changes, model upgrades, incidents — all routed there. No "AI center of excellence" maintains every agent in a shared queue; that scales to about three agents.
Change review for the tool surface. Adding a new tool to an agent goes through the same code review as adding a new API endpoint. The reviewer asks "what's the worst the agent can do with this?" before approving.
Model-version pinning and canary rollout. Same discipline as any production service. Model upgrades go to 1% of traffic, then 10%, then everyone. We've caught real regressions this way; the "just upgrade to the latest model" pattern is a production incident waiting to happen.
Compliance mapping to frameworks you already operate under. We mapped agent controls to existing SOC 2, NIST, and GDPR control families. Don't invent new compliance categories for AI; the work is to extend what already exists.
What I got wrong
Three things, honestly:
Overinvested in prompt-level guardrails before fixing the IAM layer. First quarter, I spent more time tuning prompts to refuse certain requests than I did scoping the agent's IAM role. The right move was the opposite — tighten the IAM scope by ninety percent, treat the prompt as a behavior layer, not a security layer. The lesson generalizes: when in doubt, move the control as far down the stack as it can go.
Underbuilt the audit trail. Started with logging that captured the prompt and the final answer. Didn't capture intermediate tool calls, tool arguments, model version, or request lineage. Reconstructing incidents was painful for the first few months; re-instrumenting later was painful too. The honest answer is to overbuild the audit trail on day one — it's cheap to capture, expensive to retrofit.
Treated multi-agent orchestration as harder than it was, then easier than it was. First thought: multi-agent will need a graph engine, a planner, a memory store. Second thought, after using LangGraph for a quarter: it just needs an orchestrator with explicit tool allow-lists per sub-agent. Third thought, after a cascading failure mode: it needs that, plus a hard cap on agent-to-agent calls per request, or you get cascades. The right model is explicit, bounded, observable — not let the agents figure it out.
The interesting AI safety conversation at the enterprise scale isn't about model alignment. It's about whether your AI agents are subject to the same operational discipline as the rest of your production systems — and whether your platform team is staffed to hold that line. The model is one component in a system that also has identity, audit, observability, change management, and human approval gates. Get those right and the model behaves more or less responsibly. Get those wrong and no model behaves responsibly enough.
That's not a model problem. It's a platform problem.