For the past few years, enterprise AI conversations have been dominated by optimism: bigger models, more pilots, faster automation. The prevailing assumption was simple — pick the right AI platform and progress would follow.

Reality has been far less forgiving.

Most IT leaders have discovered that production AI is significantly harder than early experimentation suggested. The real work begins not when a model performs well in isolation, but when it must operate inside environments that are secure, observable, and operationally durable.

Recent research my company conducted with enterprise cloud architects and IT decision-makers confirms what many engineering teams already know instinctively: experimentation is easy. Operationalizing AI reliably, repeatedly, and at scale is the hard part.

Once AI begins influencing real workflows, recommending decisions or triggering actions, the model quickly becomes the least interesting part of the system. The pressure shifts to everything around it.

Agentic AI is scaling faster than the environment around it

The data leaves little room for debate: AI has already moved into operational territory. Nearly three-quarters of respondents report actively training machine learning models, and 76% are running GPU workloads in production. More than 70% are investing in AI reasoning, decision optimization and AI assistants designed to execute tasks.

These are not exploratory use cases. They shape workflows, customer experiences, and internal decision-making.

Yet many of these systems are being deployed into cloud environments that predate agentic AI entirely. Nearly all organizations report that their machine learning pipelines require migrating more than 25% of their data — an early warning signal that existing infrastructure was never designed for reproducible model operations, standardized feature pipelines, or consistent policy enforcement.

In practice, agentic AI is being layered onto platforms optimized for application deployment, not governed execution-level intelligence. That architectural mismatch is where friction begins.

Governance gaps become visible under execution pressure

Governance gaps are easy to overlook during experimentation. In execution environments, they surface immediately.

Nearly all organizations store and process personally identifiable information, and most operate under regulatory regimes such as HIPAA or GDPR. At the same time, roughly half rely on public AI tools, while fewer than a quarter report enterprise-wide, governed AI deployments built on a shared framework.

This creates structural tension. AI systems are influencing production decisions inside environments where governance is inconsistent by design. Data flows through models without uniform audit controls. Policy enforcement varies across cloud accounts, teams, and regions.

This is not a tooling failure. It is a systems design failure.

When agentic AI participates directly in execution paths, it inherits the enterprise’s regulatory and operational obligations. If the underlying cloud architecture was not designed with AI-native governance in mind, teams are forced to retrofit controls into systems that were never meant to carry that load.

Multicloud complexity amplifies the challenge

Very few enterprises operate in a single cloud. Many manage between six and 20 cloud accounts across providers, with infrastructure-as-code practices that vary by platform and teams running AWS CloudFormation and HashiCorp Terraform side by side.

Devops organizations already shoulder significant operational burden, particularly around monitoring and reliability across distributed systems. Introducing agentic AI adds new stateful components, data dependencies, and life-cycle requirements. Model retraining, feature store updates, and inference endpoints must now align with identity, logging, and compliance controls across environments.

The friction teams experience rarely comes from any single AI system. It emerges from the interaction between agentic workloads and cloud estates assembled incrementally over years of modernization. The more fragmented the environment, the harder it becomes to enforce consistent governance at the AI layer.

It’s not just build vs. buy, but architectural fit

Much of the industry still frames agentic AI adoption as a build-versus-buy decision. The survey reflects heavy reliance on vendors and service providers, driven by skills scarcity and compressed timelines. But that framing misses the real issue.

The decisive question is architectural fit.

External platforms can accelerate delivery. Internal teams bring deep system and data context. What determines success is how AI initiatives integrate into the surrounding cloud environment.

When third-party capabilities are introduced without alignment to internal standards, fragmentation accelerates. But when AI systems are developed in isolation from core governance frameworks, architectural drift compounds quietly over time.

In response, many organizations are converging on a different model. Instead of isolating AI projects in silos, they are embedding external AI expertise directly inside internal delivery environments. Models are built and tested against production-grade governance from day one. Infrastructure, compliance, and observability are treated as first-class requirements, not cleanup work.

This approach recognizes that few enterprises have every AI capability fully staffed in-house, while preserving the architectural coherence required to scale sustainably.

Execution-level AI requires execution-level environment design

Agentic AI has decisively crossed into execution. Enterprises are training models, running GPU workloads, and embedding intelligent systems directly into operational workflows. At the same time, many are still modernizing pipelines, closing security gaps, and working toward consistent governance across increasingly distributed cloud estates.

The friction organizations encounter is rarely algorithmic. It is architectural.

Cloud environments built for application deployment are now being asked to support governed, reproducible, execution-level AI systems. That transition does not happen accidentally. It requires deliberate environment design.

Models unlock potential. Architecture determines whether that potential survives contact with production. As AI continues to influence real decisions and real workflows, the durability of the surrounding platform, not model novelty, will determine who scales successfully and who stalls.

—

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.