Context is becoming the missing layer in enterprise AI

The enterprise AI market is entering a new phase. For the past several years, the focus has been on larger models, faster inference and broader deployment of generative AI capabilities. Yet despite growing investment, many organizations continue to struggle with governance, accuracy, operational scalability and measurable business outcomes.

The challenge is becoming increasingly clear: Better models alone do not guarantee better results.

According to theCUBE Research, organizations are rapidly moving from AI experimentation to production deployments, but many are discovering that the gap between what AI models can do and the value they actually deliver remains stubbornly wide. Increasingly, the conversation is shifting from model performance to context.

In the latest episode of the AppDevANGLE podcast, theCUBE Research’s Paul Nashawaty spoke with Molham Aref, founder and chief executive officer of RelationalAI, about the growing AI value gap, the importance of contextual intelligence, and why the next wave of enterprise AI may depend more on relational understanding than model advancement.

Why bigger models aren’t closing the AI value gap

One of the central themes of the discussion was that many enterprise AI initiatives are struggling because they lack sufficient business context.

While large language models have proven highly effective in domains such as software development and document-centric workflows, they often fall short when asked to support business-critical decision-making processes. Supply chain optimization, pricing decisions, risk management, fraud detection and operational planning all rely on structured business data and complex relationships that extend far beyond text.

“The gap really exists in things that drive your business,” Aref explained. “Does anyone really have agents driving their supply chain? Agents deciding how to price products? There is very little evidence that’s happening at scale.”

The issue isn’t necessarily model intelligence. It’s that enterprise decision-making depends on systems of record, transactional data, business rules and relationships that are difficult to capture through prompts alone.

Context needs to be more than documents

The industry has increasingly embraced the idea that AI requires context. From retrieval-augmented generation to vector databases and semantic search, vendors across the market are racing to provide more relevant information to AI systems.

Most approaches still fall short because they focus primarily on documents and text, according to Aref.

“What is context?” he asked during the discussion. “Historically, context has included data. But more and more people are realizing the importance of semantics — how data is computed, how businesses define concepts and how important relationships are derived.”

RelationalAI’s view is that context must become executable. Rather than simply providing documents or static information, AI systems need access to relational structures, business logic and semantic models that represent how organizations actually operate.

This approach moves context from passive reference material to an active component of reasoning and decision-making.

Structured data becomes strategic

For decades, enterprise software has been built around structured data. Databases, ERP systems, CRM platforms and transactional applications remain the systems that run businesses every day. Yet much of the current AI ecosystem remains optimized for text.

This disconnect is one of the primary reasons enterprises struggle to operationalize AI beyond pilot projects, Aref explained.

“It’s really in the name — Relational AI,” he said. “The context has to be relational. Putting everything into text and documents is not enough.”

As organizations look to move from copilots to autonomous or semi-autonomous decision-support systems, relational data, semantic relationships and executable business logic become increasingly important. AI systems need to understand not only what data exists, but how that data relates to business objectives, processes and outcomes.

Cost efficiency requires better context

Context is not only a quality problem; it is becoming an economic one. As enterprise AI deployments scale, token consumption, infrastructure costs and inference expenses are becoming board-level discussions. Organizations are increasingly looking for ways to improve model effectiveness while reducing operational costs.

Aref pointed to context as a critical lever for controlling AI economics: “Having the right kind of context that very quickly guides agents to the right information and prevents them from wading through the wrong information is very important,” he said.

Rather than forcing models to repeatedly process large volumes of irrelevant information, contextual systems can guide agents toward the right data, the right tools and the right decision pathways. The result is improved efficiency, lower token consumption and potentially better business outcomes.

The next phase of enterprise AI

Looking ahead, Aref believes the market is moving toward a future in which context, semantics and specialized reasoning capabilities become foundational components of enterprise AI architectures.

The industry’s focus is gradually shifting from asking whether models can generate answers to determining whether they can generate valuable business outcomes. That distinction matters.

As organizations seek measurable ROI from AI investments, success will increasingly depend on their ability to connect models with structured enterprise knowledge, business processes and decision frameworks. 

The winners in the next phase of AI may not be those with the largest models, but those that can provide the richest understanding of how businesses actually work.

As Aref put it, “Everyone recognizes that these models on their own, without context, don’t work.”

For enterprises seeking to close the AI value gap, context may ultimately become the most important layer in the stack.

Here’s the full conversation with theCUBE Research’s Paul Nashawaty and RelationalAI CEO Molham Aref, part of the AppDevANGLE podcast series:

Image: SiliconANGLE