The AI Gold Rush

getty

For the past three years, the AI industry has been in the model building and training race. This meant bigger models with bigger clusters and more compute, demanding greater amounts of data, compute capability and budget. This race is primarily focused on claiming a stake in the biggest IT movement of the past few decades, a new gold rush.

However, as models start to converge on capability and AI platform vendors now look to restrict or manage access to its most powerful, and expensive, models, these companies are shifting from grubstake market claims to a more business-oriented and brass tacks needs of billing and metering access to those models. AI companies are starting to look more like traditional cloud computing companies than cutting-edge AI research labs.

The next AI battleground is inference, the act of running trained models in real products, for real users, millions or billions of times a day. The AI race is shifting from increasing power and capability to affordability, privacy and energy usage. Recent market reporting indicates that investors are tracking a move from training-heavy demand toward inference, where autonomous agents, enterprise copilots and always-on AI services create constant AI consumption.

Inference Is The Long Term Game

Big, highly capable frontier models take months to train and billions of dollars in hardware, power, networking and talent. Yet once that model ships, the focus is shifted to generating revenue from operating that model, something known as inference. Every prompt, query, code completion, image edit, customer-service answer and agentic workflow carries an inference cost.

The AI industry’s two phase approach to model development and operationalization is very similar to the approach of cloud computing companies. These companies, with very similar data centers invested heavily in data center development that can handle large, variable loads. Once the big capital investment is made, the shift is to generating revenue through consumption-based pricing. In this way, AI is nothing new.

Training resembles a high-intensity capital project. Inference resembles a utility meter. The former rewards intense research and development, while the latter rewards distribution, uptime, latency, procurement discipline and ruthlessly engineered cost per token. Once AI becomes a service that people use all day, they aren’t merely competing on model quality. They are competing on where inference happens, how much it costs, who governs it, how fast it responds and whether it fits into systems that companies already use.

That is why the market’s center of gravity is moving. McKinsey has estimated that inference will overtake model training as the dominant AI data-center workload by 2030, accounting for more than half of AI compute and roughly 30% to 40% of total global data-center demand. Realizing the shift from model development to model inference, Nvidia now markets its Blackwell GPU around total cost of ownership for inference, claiming full-stack optimization can cut inference TCO by up to 35 times.

The New Moat Is Cost Per Answer

The challenge for AI models is that many models are directly interchangeable with each other to get similar results. Models accept prompt inputs, whether through chat or API-based interfaces, and provide general outputs that can be used in a wide range of applications. This makes AI models a lot less stick and a shallower competitive moat, unlike cloud computing companies that have largely made it much more difficult to switch due to vendor-specific integrations with data storage, processing and functional capabilities.

This reality favors customers that can route AI work between different cloud and local models. A simple task does not need the most expensive model. A sensitive request may need local processing. A high-value enterprise workflow may justify a larger model, but only if the output drives measurable savings or revenue. As AI model inferences gets more expensive, organizations will be motivated to match the job to the cheapest system that can complete it well.

AI companies need to pay more attention to the greater ecosystem, beyond just model capabilities. This mirrors what happened in the cloud computing space, where early entrants focused just on raw storage and computing capabilities. Over time, these platforms matured and went beyond simple infrastructure-as-a-service capabilities to sell reliability, developer tooling, security, data services, billing systems and enterprise contracts. AI inference is following that route. The model is one layer. The operating system around the model may capture much of the value.

Nvidia appears to understand this. Its March 2026 GTC messaging stressed inference as a major revenue opportunity, with Reuters reporting that Jensen Huang pointed to a possible $1 trillion opportunity for Blackwell and Rubin AI chips through 2027. The company’s broader push into “AI factories” positions its infrastructure not only as a training engine, but as a production system for tokens, agents and applications. Factories win by producing at scale, with low defects, high throughput and controlled input costs, not just a single well-crafted product.

Edge And Local AI As A Strategy

The inference shift also explains the surge of interest in local and on-device AI. Running every request through a cloud-based data center is expensive. It can add latency and create privacy concerns. It can strain networks and data-center power supply. For many tasks, the cheaper answer is to move the model closer to the user.

Apple has made this logic central to Apple Intelligence. The company says its system is integrated into iPhone, iPad and Mac through on-device processing, with more complex requests routed to Private Cloud Compute running on Apple silicon. Apple’s iPhone privacy guide says requests are analyzed to determine whether they can be processed on device, and that data sent to Private Cloud Compute is not stored or made accessible to Apple.

Beyond privacy, local models focus on the economics of local inference. A device that can handle routine inference locally reduces cloud demand. It lets the platform owner decide which workloads deserve expensive server-side models. In a world of billions of AI interactions, those routing decisions are financial decisions. And beyond just coding, conversational and agentic tasks, local models are increasingly necessary for implementation in phones, PCs, cars, cameras, robots and industrial machines that can’t depend on giant remote models for every task.

The Utility Wars Begin

The first phase of AI focused on model ability. The next phase will focus on unit economics. Who has the lowest cost per useful result? Who can serve more requests with lower memory and GPU requirements? Who can run small models at the edge, large models in the cloud and specialized models in the workflow without making the user care? Who can compress, cache, batch, route and govern inference better than rivals?

The AI gold rush is not over in the sense that capital will stop flowing. It is over in the sense that the rush to dig and find the easy gold is ending. Building a high-powered dazzling model is no longer enough. One could argue that most of the models out there are already good enough for most of the tasks we’re using, and that incremental improvements will provide more benefit than exponential breakthroughs. The larger prize now sits in the utility layer, where intelligence is delivered like power, bandwidth or cloud storage. This is where AI companies now focus on providing models that are metered, optimized, embedded and constantly consumed.