In 2024, we talked to AI. In 2026, AI is talking to our systems, our customers, and increasingly, acting on our behalf. With AI agents, we are moving AI from a tool to an actor, from assistance to agency and from outputs to actions. And that changes the nature of risk. AI agents plan, execute, and interact with the world on our behalf. They send emails, move data, trigger workflows, and increasingly operate across systems without human intervention.

From Getting Things Wrong to Doing Things Wrong

Until now, risk was constrained by human limits, our speed, our access and our authority. Even when AI made mistakes, a human still had to act on them. There are many famous examples of that going wrong, for example South Africa’s recent retraction of their AI policy because of fictitious sources cited, or cases where generative AI produced hallucinated content. However while the impact may have been severe, it was bounded by the human in the loop.

With AI agents, that boundary disappears. And the risk moves from wrong advice to rapid large scale errors with real-world consequences.

AI agents interpret intent, operate under uncertainty, and make trade-offs in environments filled with incomplete or manipulated data. As these systems extend into physical domains such as “physical AI”, robotics, infrastructure and healthcare, we are no longer just guarding against financial loss or bruised reputations, but the real threat of physical catastrophe and the loss of human life.

This shift toward “artificial agency”, delegating decision-making authority without equivalent accountability, is now widely recognised as a core governance challenge.

Why Traditional Governance Breaks

Most governance models were never designed for this. They are rooted in agency theory, which assumes a human decision-maker acting on behalf of another or a corporate principal. Governance works in this model because most humans have intent, awareness, and the ability to understand and respond to incentives, accountability, and consequences. These assumptions break down in the context of AI agents. Machines do not internalize accountability or adjust behavior based on reputation or sanctions. This creates a fundamental mismatch. Agency theory is not wrong, but it assumes a type of agency that autonomous systems simply do not possess.

The Shift to Decision Authority

In the 2026 paper When AI Agents Act: Governance, Accountability, and Strategic Risk in Autonomous Organizations, the author explains that AI agents shift organizations from decision-support to decision-authority systems, and that is where existing governance models begin to fail (Chinnaraju, 2026).

Traditional systems helped humans decide; AI agents are the decision-makers. By operating autonomously at machine speed across multiple domains, these agents transform decision-support into decision-authority. This creates a critical shift in governance: organizations must now manage outcomes without controlling the day-to-day decisions that create them.

This creates a systemic governance gap, not just a technical risk. Accountability becomes unclear; oversight becomes ineffective because decisions are continuous rather than episodic; delegation becomes persistent and control mechanisms become too slow. There is no longer a clear moment where a human can intervene, approve, or even fully observe what is happening.

Why “Human in the Loop” Is Not Enough

For years, “human-in-the-loop” has been the default answer to AI risk. But this assumes that there is a clear decision point, that humans have time to intervene and they have enough context to act. In an agentic environment, none of these assumptions hold.

Perhaps most importantly, risk is no longer event-based but accumulative, driven by thousands of small, continuous decisions rather than single, auditable actions. This introduces new forms of exposure such as authority drift, objective misalignment, and control latency.

By the time a human intervenes, the agent may have already triggered the workflow, moved the data or executed transactions. Human oversight has become too slow, too late, and increasingly ineffective.

Human-in-the-loop means the human is the decision-maker. Human-on-the-loop means the human is the manager of the decision-maker. As AI agents move from tools we use to actors we supervise, understanding this shift is critical for any organization trying to balance machine speed with human responsibility.

The ultimate goal of AI governance is not to keep a human tethered to every decision, but to design a Decision Architecture where the agent’s authority is designed into its runtime environment.

A New Attack Surface: The Decision Engine

Layer on top of those attackers who are adapting. They know that our workforce is no longer just human, but humans coexisting within an AI ecosystem. This provides an even larger and more attractive attack surface. Techniques exist specifically designed to manipulate how agents perceive, reason, and act, such as hidden instructions embedded in content, poisoned data sources, indirect prompt injection and agent hijacking. The goal is no longer just to breach systems, but to influence decision-making within the AI systems themselves.

From Governance to Runtime Governance

One of the biggest gaps in current models is that governance is often treated as a "point-in-time" or pre-deployment gate, perhaps followed by an annual audit. But while governance might happen once a year, agentic AI risk happens every millisecond during execution.

Traditional risk management - manual, iterative, and episodic - is simply inadequate for this new reality. We need to move beyond "gatekeeping," code signing, and basic role-based security. Because agents are dynamic, our governance must be too. This requires a fundamental shift toward runtime, structure-based governance.

To manage agents effectively, we must treat them as organizational actors rather than static tools. Accountability can no longer be anchored in a one-time approval; it must be anchored in how authority is delegated, how objectives are defined, and how behavior is monitored in real time. Effective governance in the age of the agent is not a pre-deployment checklist, it is a continuous, automated, and dynamic architecture that can intervene, override, or revoke authority the moment an agent’s behavior drifts from its charter or as risk emerges.