AI’s so-called “messy middle” may have hit its biggest mess yet. 

As a global hardware crunch clashes with massive demand, AI costs have skyrocketed. In a scramble for more compute, AI services companies have had no choice but to raise their prices and revise their billing models, throwing budgets out of balance and derailing innovation just as many organizations were finally hitting their stride. 

The result is more than sticker shock: it’s an existential crisis for a business world that effectively just had one. Companies have overhauled their roadmaps to accommodate the impact of AI. Teams have rearranged their workflows in line with AI capabilities. AI enablement has become organizations’ primary competitive differentiator. AI swung into the field, kickstarting massive transformation, and now teams are being asked to pull back when they’ve only begun to operationalize it.

Companies like Uber have made headlines for placing aggressive caps on AI usage to stem the flow. But while usage limits can be an effective cost-cutting measure, they’re not the only way that businesses can cut back on computing spend.

The first place they should be looking is their cloud bill. 

It seems counter-intuitive. While the formula has always been that more compute power equals more AI impact, that doesn’t necessarily make it more efficient. Enterprises are hemorrhaging their budgets to AI services while still relying on oversized, overpriced infrastructure; rising AI costs are simply exposing a deep-seated problem that enterprises can no longer ignore. 

How cloud spend accumulates

IT infrastructure spend is often an organization’s second-biggest line item after headcount, accounting for an average of 10% of a business’s annual revenue – a figure that AI demand has pushed skyward in the past few years. But many are adding volume to an already overbloated IT framework.

CTOs and other technology decision makers are drawn to what’s familiar and reliable. Unfortunately, this is also what’s spiking their cloud bills. By embracing the tried-and-true, they’re getting locked into expensive one-size-fits-all deals with major hyperscalers where they pay for solutions that don’t generate value.

Over time, they find they’re paying for unused or inactive services, unoptimized hardware and empty storage, when they could be investing that money into AI innovation. With AI only getting more expensive, they’ll need every dollar they can get. 

Rightsizing infrastructure for AI efficiency

Getting cloud sprawl under control requires a combination of acute, short-term action and long-term evolution. Tackling what’s solvable today gives enterprises the breathing room to plan for bigger readjustments over the next year. 

Transitioning to new infrastructure takes time, and organizations need solutions now so their teams can keep working with AI. Here are three steps enterprises can take immediately to modulate their cloud spend:

1 – Analyze CPU and memory usage

There is such a thing as “too much” performance. IT teams should conduct a thorough diagnosis of cloud services usage, then downgrade instances that exceed performance requirements. A deep dive into cloud usage also helps teams discover features and infrastructure that aren’t being used at all; if their current contract allows, they may be able to decommission these services or negotiate a lower payment throughout the remainder of the contract. Otherwise, identifying unused features provides a valuable framework for determining the parameters of their reconfigured cloud approach. 

As enterprises shift their infrastructure strategy, they’ll want to opt for cloud services that economize memory usage. For example, memory-optimized infrastructure and vCPUs deliver more affordable support for enterprise-grade support compared to the Arm-based optimizations typically offered by hyperscalers. This relieves some of the monitoring burden from IT teams while simultaneously decreasing overall cloud spend. 

2 – Set strict spending limits

Spending and usage limits have been the tech industry’s immune response to AI price hikes, but it’s not just setting the limits that matters. It’s how those limits are defined, communicated and enforced. 

For one, spending limits should be use-specific. Leaders should be able to use them as a guide for project planning and resourcing, allowing them to structure budgets, timelines and individual project teams with a mind toward efficiency. 

As for communication, it’s crucial that the entire workforce understands the why behind the limits. We are used to seeing the digital as the infinite, but at this phase of the AI boom, we’re up against the laws of physics. As shortages and billing changes impact employees’ day-to-day workflows, spending limits should be positioned as a sustainable way to maintain their new reliance AI-powered tools, rather than a restriction. 

3 – Enforce tagging protocols  

Tagging protocols prevent unmanaged IT use that sneakily drives up computing expenses. With a robust tagging code, IT leaders can gain a better understanding of cloud services traffic, allowing teams to target the source of overuse and make strategic adjustments to cut resources where bloat is happening. At the same time, tagging protocols can also show where teams may need an upgrade. 

As their existing cloud contracts expire, enterprises have the opportunity to restructure their computing ecosystem with a composable, multi-cloud approach. This doesn’t necessarily mean abandoning their longtime hyperscaler partners, but downsizing their contracts to harmonize with more efficient infrastructure elsewhere in the stack. Alternative clouds, edge solutions that offload the strain on GPU-based infrastructure, and open-source software solutions all comprise a flexible, multivendor strategy that does more than simply save costs: it increases the ROI of every AI initiative by maximizing price for performance, allowing enterprises to then invest the savings back into AI innovation. 

Rewiring the efficiency mindset

Managing AI spend is also a culture issue. AI-enabled workforces are notably obsessed with efficiency. That’s why they’re building agents, automating their communications, and letting AI do the grunt work while they expand their capacity for creativity and innovation. But irresponsible use of these tools has the opposite effect. 

“Tokenmaxxing” and other trends that encourage wasteful AI use are incompatible with a bias to efficiency. If teams want to actually do more with less, their interactions with AI should reflect that principle. 

That requires shifting culture and technology in tandem. Encourage the use of better prompts to reduce inference load, but also adopt infrastructure that automatically economizes inference. Adopt smaller AI models with precision use cases. Embrace open source models and software to reduce training workloads. Empower developers to solve problems efficiently and educate non-technical AI users. Companies that want to encourage their employees to use AI must do so responsibly. 

Despite price hikes, we haven’t hit the brakes on AI transformation. Operationalizing AI at scale was always going to be an expensive endeavor, requiring careful planning and strategic capital allocation to remain sustainable. Where we are is a pinch point: the moves IT leaders make now will determine who passes through, and who gets left behind. 

While the industry mobilizes to solve the brewing compute crisis, enterprises need to find ways to make do with what they have. Companies can protect themselves against astronomical AI spend by addressing overspending on cloud solutions, economizing inference across a distributed cloud, and investing in infrastructure that promotes real ROI.