Is the promise of AI productivity really true?

There's a story the AI industry loves to tell: give anyone access to AI, and they'll be more productive. Doesn't matter who they are, what they know, or how much experience they have. The tool does the heavy lifting. You just steer.

It's a great story. It's democratic. It's empowering. And the research increasingly shows it's not that simple, especially when it comes to working with AI in real-world situations.

In 2025, a landmark study tracked 5,172 customer support agents at a Fortune 500 company who were given access to a generative AI assistant. The average productivity gain was 15% - a solid number.

But the average hid a much more interesting story.

Less experienced agents saw massive improvements. They resolved more issues, faster, with better quality. The AI essentially functioned like a senior colleague whispering the right answer in their ear.

The most experienced agents? Small gains in speed. And small declines in quality.

That finding didn't get the headlines. But it should have. Because it tells you something important about what AI actually does: it closes the gap between novice and competent. It does not automatically push competent toward exceptional.

A Harvard Business School study on AI made this even sharper. Researchers gave marketing specialists, software developers, and web analysts access to AI tools and asked them to perform each other's jobs.

AI helped everyone get started. It was great for conceptualization - generating ideas, framing problems, and structuring an approach. But when it came to execution - the detailed, context-dependent, hands-on part of solving problems with AI - people without domain experience still produced noticeably worse output.

Marketing specialists with AI couldn't match the quality of web analysts doing their own job. The technology filled knowledge gaps, but it couldn't replace the lived experience of actually understanding the domain.

As the researchers put it: AI can provide the map. But navigating the terrain is another matter.

Here's the finding that should make everyone uncomfortable.

A randomized controlled trial published in 2025 tracked 16 experienced open-source developers completing 246 tasks on codebases they'd worked on for an average of five years. Half the tasks allowed AI tools; half didn't.

Before starting, the developers predicted AI would reduce their completion time by 24%. After finishing, they estimated it had saved them about 20%.

The actual result? AI increased completion time by 19%.

The tool made experienced developers _slower_, not faster. And they didn't even realize it.

There's a pattern emerging across all of this research, and it points to something the productivity discourse consistently gets wrong.

The bottleneck for most knowledge work isn't production. It's judgment.

AI is extraordinarily good at production. It can generate text, code, analysis, summaries, and drafts faster than any human. It can produce a hundred options where you might have considered three.

But the value of those options depends entirely on whether you can evaluate them - rather than relying entirely on AI decision-making.

Can you tell the difference between a good strategy and one that just sounds good?

Can you distinguish technically correct code from code that will fail at scale?

Can you recognise when AI-generated marketing copy misses the tone entirely?

That's judgment. And it comes from experience, domain knowledge, and an understanding of context that AI doesn't have.

The high-performing entrepreneurs used AI to generate options, then applied their own judgment to select the right ones. The low performers took AI's suggestions at face value. Same tool. Same access. Wildly different outcomes.

The difference wasn't the AI. It was what the human brought to the table.

This is the core problem with the "AI makes everyone more productive" narrative.

Speed is only valuable if you're going in the right direction. AI has no inherent sense of direction. It has no judgment about whether the output it produces is actually good for your specific situation. It just produces.

Give an experienced product manager AI tools, and they'll explore options faster, validate assumptions, and stress-test decisions. They get better, faster because they already know what good looks like.

Give the same tools to someone without that foundation, and they'll produce more output. It might look polished. It might be grammatically perfect and formatted beautifully. But it may be strategically wrong, technically naive, or contextually off-target in ways that only become visible after the damage is done.

A survey of 2,500 professionals found that 77% reported AI actually increased their workload. Nearly half said they didn't know how to unlock the productivity benefits. The tools were there. The judgment to use them well wasn't.

There's a particularly dangerous zone that nobody talks about: when AI output is almost right.

When AI produces something obviously wrong, you catch it. When it produces something excellent, you benefit. But when it produces something that's 85% correct - coherent, reasonable, plausible - that's when the real risk lives.

Because it takes more expertise, not less, to identify what's subtly wrong.

And the confidence AI output carries - clean formatting, authoritative tone - makes it harder to question.

This is where many people struggle. Not because they lack intelligence, but because they lack the experience needed to evaluate something that looks correct.

AI doesn't solve this problem. In many cases, it makes it worse by producing a higher volume of plausible-but-flawed output that overwhelms the judgment capacity of the person reviewing it.

None of this is an argument against AI. It's an argument for understanding what it actually does.

AI is an amplifier. It takes whatever you bring to the table - your expertise, your judgment, your understanding of context - and it multiplies it.

If you bring deep knowledge, it makes you more effective.

If you bring shallow understanding, it makes you more efficiently shallow.

This has real implications for how we use AI for work.

It means:

- expertise matters more, not less

- judgment is the real leverage

- tools alone don't create results

The AI productivity conversation has been dominated by tools, models, and capabilities: which model is fastest, which tool has the best features, and which prompt technique unlocks the most value.

But the research keeps pointing to the same unsexy conclusion: the variable that matters most is the human.

Not as a prompt engineer.

Not as an AI operator.

But as someone with the experience and judgment to direct outcomes.

AI won't make you faster if you don't know where you're going.

It will just get you to the wrong destination sooner.

The idea that AI democratizes expertise is appealing.

But in reality, it amplifies whatever expertise already exists.

That's where the gap and the disappointment come from.