AI Infrastructure Scarcity is Raising Costs, but AI Usage Will Still Provide Unbeatable ROI
Costs will continue to rise, likely a lot, from inescapable scarcity challenges over the next few years. Luckily, AI capabilities will rise faster. Every dollar of AI compute will return 70-400% **more* useful work within two years -- even after scarcity takes its cut.*
AI infrastructure is fighting headwinds. Memory prices up 60%. Power grids buckling. Chip manufacturers sold out through 2026 and GPU rentals maxing out. The word "scarcity" dominates every AI infrastructure conversation. And yet: the math still favors investment, decisively.
Two Cost Curves, One Critical Distinction
AI costs rest on two concepts that people mistakenly conflate:
Cost per token -- the raw unit cost of AI compute -- analogous to the price of a gallon of gas. This has dropped ~40-50x per year for the past 5 years [1] [2]. Staggering.
Cost per useful task -- what you will spend on a task that uses tokens -- for example, the all-in cost of driving from New York to Boston (yes, there ARE New Yorkers who want to go to Boston!). This has improved only ~2-4x over two years [3]. Still impressive, but a very different number.
Why the gap? Because the tasks we elect to spend on are evolving, too. AI usage has evolved from simple chatbot queries (500-1,000 tokens) to agentic workflows -- multi-step processes where models reason, code, test, and iterate (50,000+ tokens per task is common) [4]. In our analogy, gas prices may be cheaper, but now we're traveling in a flying car. It takes a lot more gas for the same old tasks, but we can also use it for things we couldn't use it for a year ago. Long-running tasks and larger context windows both allow that flying car to do a great deal more, even when it might not be necessary. It's sometimes like bringing in a math PhD to do elementary school homework -- you'll get the same answer, but a lot more horsepower is going into it. In the AI case, you pay for that horsepower, and over-provisioning inefficiency is rampant (I predict that tools which dispatch the right questions to the right models will become universal this year...smart dispatch for right-pricing the work).
The token price is the headline. The task cost is what hits your P&L.
The Agent Swarm Multiplier
There's a further capability gain the 2-4x doesn't capture: multi-agent coordination. On SWE-bench Verified, multi-agent systems resolve 76% of software issues versus 43% for single-agent approaches -- a near-doubling [5]. Goldman Sachs projects 3-4x productivity gains from deploying thousands of coordinating agents [6]. Harvard Data Science Review estimates 2-10x gains from agent-centric workflows, with the range depending on whether organizations redesign their processes or just bolt agents onto existing ones [7].
Gartner reports a 1,445% surge in multi-agent inquiries and predicts 40% of enterprise apps will feature task-specific agents by end of 2026 [8]. Anthropic and Microsoft both frame coordinated agent teams as the default model within 12-18 months [9][10].
Factoring in swarm gains, the task improvement range extends to 2-6x over two years (and our experimentation with agent teams/swarms are showing much higher gains than this). Note, however, that achieving the high-end multiplier of 6x requires adopting multi-agent workflows, not just single-agent tools. Regardless, every year is bringing more bang for your buck.
The Scarcity Headwinds Offset
On the supply side, however, the constraints are real and physical:
Power: Seven-year queues for data center connections in Northern Virginia [11]. PJM (Mid-Atlantic/Midwest power for 65 million people) capacity prices jumped 11x in one year [12]. Data center GPU occupancy rates at 95%+ [13].
Memory: Every major manufacturer has pre-sold its 2026 output. 70% of global production flowing to AI. Prices up 60% [14]. New fabs take 2-3 years to build.
Chips: NVIDIA Blackwell and AMD MI series allocated well into 2026+ [15]. TSMC concentration risk (one island with geopolitical tension) sits behind all of it.
The bill: McKinsey estimates $6.7 trillion in cumulative data center investment needed by 2030 [16].
For cloud/API buyers (most enterprises), providers absorb much of this pressure. Expect a 1.5-2x cost increase versus the no-scarcity trajectory. Since prices have been declining, this means they will decline more slowly (rather than rising from today's levels) [17].
For in-house buyers, every factor lands directly on your P&L. A 2-3x increase in infrastructure costs is plausible.
The Net Math
Divide the task improvement by the scarcity headwind to see an estimated increase in net work per dollar.
Cloud/API buyers
- Optimistic: [scarcity increase 1.5x] x [Task Improvement gain 2-6x] = 1.3-4x $
- Base Case: [scarcity increase 2x] x [Task Improvement gain 2-6x] = 1-3x $
In-house Buyers
- Base case: [scarcity increase 2.5x] x [Task Improvement gain 2-6x] = 0.8-2.4x $
- Pessimistic: [scarcity increase 3x] x [Task Improvement gain 2-6x] = 0.7-2x $
The range is wide by design. The low end would occur from adoption of single-agent, unoptimized workflows. The high end results from multi-agent coordination with model cascading and caching. Where you land depends more on what practices you adopt than what you spend.
Cloud base-case midpoint: ~2x. Your $100 of AI compute today gets you $200 worth of useful work in two years. The scarcity ate a chunk of the improvement. The improvement still won.
No other business input delivers this return profile. U.S. labor productivity averages 1.5% annual growth [18]. The S&P 500 returns ~10%/year. AI compute improvement, even scarcity-adjusted, delivers multiples of that.
Why Lease, Not Own
In-house compute has to try to balance scarcity costs against a fixed capability baseline -- your NVIDIA H100s don't get smarter. On the other hand, leased compute absorbs scarcity against a constantly improving capability curve -- when your provider upgrades models, you get the improvement bundled in.
This asymmetry widens under scarcity. For most organizations, the economic case for leased compute strengthens, not weakens.
The J-Curve: Start Now
There's a timing dimension the math doesn't capture: teams slow down before they speed up.
Brynjolfsson's productivity J-curve [19] holds for every general-purpose technology. A rigorous 2025 trial found experienced developers actually took 19% longer with AI tools initially (but they thought they were 20% more productive! I'll talk about why I think that was the case in another article) [20]. MIT Sloan found a ~2-year productivity dip in manufacturing firms before achieving dramatic gains [21]. McKinsey reports 92% of companies increasing AI spending while only 1% have reached maturity [22].
The trough isn't optional. It's the cost of building the skills and workflows that make the technology productive.
Here's why understanding the J-curve matters: the learning curve takes months to years. Every quarter you delay, you push your harvest phase deeper into the scarcity window -- when experimentation costs more, and your competitors who started earlier are already compounding gains. The cheapest time to make mistakes and learn what works for your firm is before scarcity pricing fully kicks in.
What To Do
Budget for scarcity: Assume 50-100% cost increase over trend for cloud, more for in-house. Then model the 0.7-4x capability improvement you'll receive at that higher cost.
Lease, don't own: Let cloud providers absorb the capital risk. Buy capability, not hardware.
Optimize workflows: The gap between optimized and unoptimized AI workflows determines whether you land at 0.7x or 4x. Invest in model cascading, caching, and multi-agent architecture (my team can help you up the curve -- context- and spec-driven development with test-driven development for the actual coding, brownfield strategies that actually work, agentic harnesses for speed/quality/observability, team/swarm strategies with orchestration, and so forth).
Start the learning curve now if you haven't, and accelerate to the extent possible regardless: The J-curve means you'll slow down first. Accept it. Every quarter of delay pushes your trough into more expensive territory and forfeits compounding returns.
Lock in access: Compute access is becoming a strategic asset. Secure pricing and capacity commitments while you can.
The Bottom Line
Scarcity is real. Costs will rise. And despite all of that, if these estimates are anywhere near what actually happens, you'll still get 70% to 400% more useful work per dollar within two years -- or more. The high end goes to organizations that adopt multi-agent workflows and optimize aggressively. In-house buyers face a tighter picture, making the need to invest NOW in learning and process improvement even more important, but even 70% more useful work than now is extraordinary.
No other investment delivers that return profile. Plan for the cost increase. Start learning now. And focus on being one of the organizations that captures the improvement, not one waiting for perfect conditions that aren't coming.
How is your organization modeling AI compute costs against capability improvement? Are you optimizing your agent workflows, or paying the full token tax? Are these numbers ludicrous? If you'd like help getting up the agentic coding curve, drop me a note.
References
[1] Stanford HAI, "The 2025 AI Index Report," 2025. (280x cost reduction for GPT-3.5-equivalent inference in under two years.)
[2] Epoch AI, "LLM inference prices have fallen rapidly but unequally across tasks," 2025. (Median ~50x/year cost decline; range 9x to 900x depending on benchmark.)
[3] Epoch AI / arXiv, "The Price of Progress: Algorithmic Efficiency and the Falling Cost of AI Inference," Nov 2025. (Algorithmic efficiency ~3x/year; hardware efficiency ~1.3x/year.)
[4] Xenoss, "Total cost of ownership for enterprise AI: Hidden costs and ROI factors," 2025. (Enterprise AI cost scaling; API fees as ~10% of total agent system costs.)
[5] Verdent AI, "SWE-bench Verified Technical Report," 2025. (Multi-agent: 76.1% pass@1 vs. single-agent ~43%.)
[6] Lucidate, "Goldman Sachs Scales AI Coding to Thousands of Agents," 2025. (Projected 3-4x productivity gains from agent teams.)
[7] Harvard Data Science Review, "The Agent-Centric Enterprise," 2025. (2-10x gains; requires process redesign.)
[8] Gartner, "40% of Enterprise Apps Will Feature AI Agents by 2026," Aug 2025.
[9] Anthropic, "2026 Agentic Coding Trends Report," 2026.
[10] Microsoft, "2025 Work Trend Index: The Frontier Firm," Apr 2025.
[11] Bloomberg, "Virginia Data Centers Face Seven-Year Wait for Power," Aug 2024.
[12] IEEFA, "PJM capacity prices up 10x," 2025.
[13] JLL, "North America Data Center Report," 2024/2025. (Occupancy ~97%+.)
[14] CNBC, "AI memory is sold out," Jan 2026. (Prices up 30-60%; 70% of DRAM to AI.)
[15] eWeek, "NVIDIA Blackwell Sold Out," 2024/2025.
[16] McKinsey, "The cost of compute: A $7 trillion race," 2025.
[17] Silicon Data, "H100 Rental Price Over Time." (Peak $8-11/hr; now $2-3.50/hr.)
[18] U.S. Bureau of Labor Statistics, "Productivity and Costs." (~1.5% annual growth, 2007-2019.)
[19] Erik Brynjolfsson et al., "The Productivity J-Curve," AEJ: Macroeconomics, 2021. Also: Fortune, "AI Productivity Liftoff," Feb 2026.
[20] METR, "Measuring the Impact of Early-2025 AI on Developer Productivity," Jul 2025. (Experienced devs 19% slower with AI; believed 20% faster.)
[21] MIT Sloan, "The Productivity Paradox of AI Adoption," 2025. (~2-year dip before gains.)
[22] McKinsey, "Superagency in the Workplace," Jan 2025. (92% increasing AI spend; 1% at maturity.)