Alexey Spas is the founder and CEO of Instinctools, a software engineering company focused on AI-powered digital solutions.
getty
At some point in recent months, in conference rooms and on Zoom calls, senior executives started asking a deceptively simple question: How much more autonomy are we actually comfortable giving these agents?
By now, companies already have proof that agentic AI works. One agent might be helping service teams resolve tickets faster. Another might be drafting internal reports. A third might be moving information across systems with barely any human help. And that is the moment most companies are not prepared for. Because once software begins to act, the enterprise challenge goes past a capability story to the practical realities of such delegation.
McKinsey’s latest research on AI trust backs up this claim: Nearly two-thirds of respondents say security and risk concerns are the top barrier to fully scaling actionable AI. Furthermore, almost 60% of respondents who had experienced AI incidents rated their organization’s response as merely satisfactory or outright negative.
Although AI infrastructure providers are eager to promise that humans are still in charge, such reassurance sounds less comforting than it is meant to, unless someone defines what control looks like in practice.
I'm not talking about control in the narrow sense of locks and permissions, but rather in the broader, less obvious sense that matters in real organizations: clear ownership, disciplined onboarding and retirement, orchestration, reliable guardrails, observability and a trustworthy data foundation. In my own work on GENiE, our proprietary multi-agent platform, I have seen that these elements only work when they are built and treated as a whole. And until they are, most companies will find that the path to greater autonomy is constrained less by what agents can do and more by the risks they introduce.
The first component of that control layer is the least glamorous and maybe the most important: human ownership. One reason agentic AI still feels slippery inside large organizations is that too much responsibility remains diffused. The workflow belongs to one team, the model belongs to another, and the data belongs to a third—and nobody is fully accountable for how the agent behaves over time.
It’s organizations with explicit ownership for responsible AI (particularly through AI-specific governance roles, internal audits or ethics teams) that show materially higher maturity than those without a clearly accountable function. I've seen this firsthand in my work. This is why companies need agent managers: people who monitor performance, watch observability dashboards and treat agents like systems that require active supervision.
That logic should extend from agent onboarding to retirement. One of the mistakes I see many companies make is treating agents like software features: You turn them on, connect a few tools and let them be. Unfortunately—or perhaps fortunately—autonomy doesn’t work this way.
For companies to make sound calls about which tasks can be delegated, which still require approval and when an agent has become more trouble than it is worth, a more disciplined approach is required. You need to give each agent a job description, make its role clear to human colleagues, define its boundaries, evaluate its performance over time and, when necessary, retire them.
This is also where it helps to separate control from orchestration, because the two are starting to blur in the market conversation. Orchestration matters. Multi-agent systems need coordination across tasks, tools and workflows. But orchestration answers only one question: How do agents work together? Control answers a harder one: How does the enterprise stay in charge of what those agents are doing?
That distinction is becoming more important now that vendors are positioning agents as the new interface for enterprise software, rather than a replacement for their underlying systems. Large companies are not ripping out core platforms like ERP and CRM; they are building AI-driven workflows and lightweight customizations on top of them. That makes the supervisory layer even more critical, with agents increasingly sitting above those complex systems and using their data.
The next part of the control layer is reliable infrastructure: identity, permissions, logging, audit trails and action controls. Cisco is arguing for something similar from the security side, saying enterprises need to move beyond access control to action control for digital co-workers. That is the right phrase, as the problem is no longer just what an agent is allowed to see, but what it is allowed to do and what the organization can prove after the fact.
That explains why so much of the next market will be built around monitoring the monitors. By 2030, guardian agent technologies will account for 10% to 15% of the agentic AI market. The implication is hard to miss. Part of the control layer will itself be agentic. Companies will use software not only to automate work, but to supervise the software doing the work, including blocking or redirecting actions to keep systems aligned with predefined goals.
Underneath all of this sits the least visible component of autonomy: the data foundation. Agents are only as trustworthy as the enterprise reality they are allowed to function in. The most valuable agentic applications are being built around unified enterprise data, workflows, policies, approval hierarchies, permissions and transactional context. That is a reminder that intelligent autonomy does not rest on clever prompting alone, but rather on whether the system is grounded in well-prepared data and the right business context. Without that foundation, more autonomy just means faster mistakes.
As agentic AI became the norm, many companies treated control as something to work out later, forgetting that the technology is introducing a new class of operational risks. Agentic AI will require new forms of oversight to address them. It is not enough to invoke a human in the loop. If the structure is vague, the safeguard quickly turns into a choke point. The companies that rise higher on the autonomy ladder and alchemize intelligence into revenue will be the ones that build structure into those agents strong enough to let humans stay in control.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。