When we deployed agentic AI across one of Australia’s largest tourism and cruise operators spanning B2C booking, B2B wholesale, cruise operations, offshore shared services and a live marketplace, we solved most of the expected hard problems faster than anticipated. The small language models worked. The tools integrated. We identified the right proprietary data and focused on what gave us decisions, insights, hindsight and foresight. The tech hype, to its credit, delivered.

What we hadn’t fully anticipated was governance, not the high-level policy kind, but the granular, daily, operational kind. The kind that keeps a 34% reduction in Tier 1 support escalations from becoming a 134% increase the day an agent drifts. The kind that determines whether a guest’s cruise booking gets silently corrupted at midnight, or caught within seconds.

Most organizations stop at implementation, then pivot to a governance framework and high-level reporting. That is not governance; that is performance review. Real governance is what happens between the reviews, and that is the gap this piece is about. Not the theoretical gap, the operational one. The one line-of-business managers, technology teams and compliance officers actually live in.

Why implementation isn’t the finish line

In traditional software, “go live” is a milestone. In agentic AI, it is the beginning of the most demanding phase. Agents, unlike static software, learn from context, adapt to signals and make decisions within defined boundaries. But those boundaries erode. Models drift. Tool outputs change. Data quality degrades. The most dangerous version of this is drift without deviation: the agent gradually shifts its decision patterns without tripping a single alarm, because the guardrail was never wrong; the tolerance window was simply set too wide. And unlike a human employee who hesitates when something feels off, an agent executes with confidence until something breaks a hard constraint.

The risks are compounding in ways that catch organizations off guard. A misrouted email costs one customer. A misrouted agentic decision can propagate across every booking, query or escalation processed in the same window. An agent acting on stale pricing data doesn’t know the data is stale; it acts with the same confidence it would on good data. Humans second-guess; agents don’t.

And when a human employee makes an error, the chain of accountability is clear. When an agent does, caught between model, tool, data and prompt it often isn’t. That accountability vacuum is where governance failures begin.

This is not a rare failure mode. Gartner expects more than 40% of agentic AI projects to be cancelled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls. The first two get argued about in steering committees long before go-live. The third only reveals itself afterwards, which is precisely why the rewire-or-rebuild decision has to account for the operating model, not just the architecture.

Guardrails, tolerance limits and how to decide them

Guardrails are only as good as the tolerance limits you set, and most organizations set them based on intuition rather than evidence. Established frameworks help you structure the problem; NIST’s AI Risk Management Framework gives you the govern, map, measure and manage scaffolding, but no framework can hand you your own numbers. Getting those right requires a deliberate calibration process drawn from actual operational data, not hypothetical scenarios.

In our cruise group deployment, we used a four-tier tolerance model. Each agent behaviour was classified by its reversibility, customer impact and financial materiality. That classification determined where the guardrail fired and how.

Naren Gangavarapu

Informational outputs sit in a wide tolerance band, log anomalies, flag them at a weekly review, but don’t interrupt flow. Workflow triggers sit in a moderate band: a human review queue, with auto-pause once a threshold is breached. Transactional actions sit in a narrow band: mandatory human confirmation, rollback protocol active. External customer communications sit at zero tolerance: no agent sends autonomously, ever.

Tolerance limits should be set collaboratively by operations, legal, risk and the line-of-business managers who understand what a bad outcome costs. Technology sets the mechanism. The business sets the threshold. Conflating the two is where most governance frameworks break down.

Educating line-of-business managers: governing their AI employees

This is where most agentic AI programs quietly fail. The line-of-business manager who runs cruise operations, manages the wholesale desk or owns the customer service floor is now accountable for both human and AI employees. But they were never trained for the latter.

You would not put a new hire on the floor without onboarding, a buddy system, performance reviews and an escalation path. Agents require the same structure, and so do the managers responsible for them. The most effective frame we found was treating agents exactly like high-volume junior team members: fast, consistent, tireless and capable of significant harm if poorly supervised. Managers responded to that framing. It made the governance conversation concrete rather than theoretical.

In practice, that meant building six governance habits into the operational rhythm of every line-of-business manager with AI employees.

Naren Gangavarapu

Critically, it also meant establishing explicit human-agent teaming norms: protocols for when a manager overrides an agent, when they defer and how that decision is logged. Override without logging is an invisible governance failure. The override itself isn’t the problem; the absence of a record is.

When things go wrong: containment, speed and customer protection

In an agentic system, failure is not a question of if, it’s when, and how fast you contain it. The goal isn’t perfection; it’s a blast radius so small the customer never feels it.

We designed a four-phase incident response with strict time targets, and speed is the primary design constraint, not thoroughness. Detection inside two minutes, by an automated anomaly alert rather than a customer complaint. Containment within five minutes, with the agent paused or rerouted to a human. Impact confirmed within 15 minutes, by checking whether the failure stayed inside the agent’s boundary. Root cause identified and a fix deployed within an hour, with the post-incident review scheduled within 24. Thoroughness comes in that review, not in the first hour.

The key design principle is boundary-first thinking: every agent must have a defined operational perimeter. When a failure occurs, the first question isn’t “what went wrong?” it’s “did the failure stay inside the perimeter?” If yes, you have time. If no, the clock is running on customer impact, and you escalate immediately.

In our deployment, the most effective containment mechanism was not technical; it was a human-in-the-loop circuit breaker that any manager could activate within 90 seconds. No ticket. No chain of command. One action. The agent stops and human routing resumes. The simplicity was deliberate: under pressure, complex procedures fail. It is also where operational instinct and regulation are converging: Article 14 of the EU AI Act requires that high-risk systems can be interrupted through a stop button or equivalent, and that a human can disregard, override or reverse an output. We built ours because we needed it on a Tuesday night, not because a statute told us to.

Compliance and organizational law: the non-negotiable layer

Compliance is not a box you check before go-live. In agentic AI, it is a living constraint that must be embedded in every decision loop the agent runs. Privacy law, consumer protection, financial services obligations and sector-specific licensing do not pause because your agent is processing at scale. The OAIC’s guidance on privacy and commercially available AI products is explicit on the point: privacy obligations attach to personal information put into an AI system, generated by it, or processed through it, and the due diligence expected of you includes assessing human oversight capability before deployment, not after.

Three compliance principles proved non-negotiable in our environment. First: delegation is not absolution; the organization remains legally responsible for every agent decision. Second: consent and disclosure travel with the agent; privacy obligations apply regardless of whether a human is in the loop. Third: audit trails must be agent-native; every decision must produce an auditable record from day one.

Australia’s Voluntary AI Safety Standard and its ten guardrails signal the direction of travel, and the EU has already set the destination. The temptation right now is to read the deferral of the EU AI Act’s high-risk obligations to December 2027 as breathing room. It is not. The compliance date moved. The liability did not. Organizations deploying agentic AI today should build for the regulatory environment of 2028, because the cost of retrofitting compliance is always higher than building it in.

Making governance part of the organizational DNA

Governance frameworks that live in SharePoint folders don’t govern anything. For agentic AI to become part of organizational DNA, the governance mechanisms must be embedded in the daily rhythm of operations, as automatic as a safety briefing, as natural as a shift handover.

The organizations that will get this right are the ones that treat AI governance not as a compliance burden added to operations, but as a new operational competency built into them. In practice, that looks like daily agent performance visible on the same dashboards as human team KPIs; governance roles assigned to existing operational leaders rather than siloed into a technology team; and a cadence of real incidents, however small, reviewed openly so the organization builds genuine intuition about how agents fail, not just how they succeed. It is also what boards are now being told to look for; the AICD and UTS Director’s Guide to AI Governance puts oversight of AI systems squarely inside existing director duties rather than alongside them.

Monthly recalibration sessions where tolerance limits are reviewed against actual incident data are the mechanism by which an organization learns from its agents. What fired that shouldn’t have? What didn’t fire that should have? These are the questions that sharpen a governance framework from theoretical to operational.

The companies that scale agentic AI successfully won’t be the ones with the best models. They’ll be the ones with the best operational habits around those models. The technology is, increasingly, a commodity. The governance maturity is the differentiator.

In our tourism and cruise deployment, the outcomes that mattered — a 34% reduction in Tier 1 support escalations, operator onboarding reduced from 23 days to 3, and a 24% uplift in booking conversion — were only sustainable because of what we built around the agents, not just in them. The technology was the easy part. The governance was the work.