The Gell-Mann Amnesia Effect of AI is a pretty well documented phenomenon:
The Gell-Mann amnesia effect is a cognitive bias describing the tendency of individuals to critically assess media reports in a domain they are knowledgeable about, yet continue to trust reporting in other areas despite recognizing similar potential inaccuracies.
Summarizing, AI sounds like an incredible genius synthesizing the world’s knowledge right up until you ask it about the thing you know about, then it’s an idiot. Even knowing about this phenomenon and having experienced it countless times, LLMs have an intoxicating quality to them.
If there’s one thing LLMs do well, it’s delivering an enormous blast of dopamine if they do a good job on the first try. That “if” does a lot of heavy lifting, but when it happens you start to believe the idea that the multi-modal chat box could make you faster and infinitely capable. A hop and a skip and hundreds of dollars in tokens later, you made something that would take lifetimes of learning to create.
The marketing around Generative AI is that every person can be their own Designer, Developer, Researcher, Content Writer, Video Producer, and Podcast Editor. With the right skills.md files everything will fall into place. We will automate all automatable tasks. The experience will be “frictionless”.
You’ll be able to automate away the jobs you don’t enjoy…
And the jobs you don’t know how to do…
And the people who do those jobs…
The people you don’t enjoy…
You won’t need to talk to or wait on anyone. You can automate away anything or anyone that stands in your way. You will reduce costs. You will be richer. It will be frictionless.
Sometimes I feel like there’s a palpable tension in the air as if we’re waiting to see whether AI will replace designers or engineers first. Designers empowered by AI might feel those pesky nay-saying, opinionated engineers aren’t needed anymore. Engineers empowered with AI might feel like AI creates designs that are good enough for most situations. Backend engineers feel like frontend engineering is a solved problem. Frontend engineers know scaffolding a CRUD app or an entire backend API is simple fodder for the agent. Meanwhile, management cackles in their leather chairs saying “Let them fight…”
I think it’s a dangerous place to be when we start to consider people as friction.
That’s because we know Gell-Mann is real. We know there’s an optimism bias the size of the sun blinding us to the actual quality of what the machine is producing. We need knowledgeable people to share what they know to improve the quality of our work, generated or otherwise. We even need ignorant people to make sure we can break ideas down into their simplest form that everyone, agents or human, understand. People can have bad attitudes, be shitty, and have wrong opinions… but people are not friction. An LLM may be able to autocorrect its way into a plausible human response, but it’s not people. It doesn’t care if it’s right or wrong. The money and hype surrounding it acts as a shield to its reputation. LLMs make up answers if it doesn’t have enough context and they fall over if it has too much context. It amplifies good patterns the same as bad patterns. And it glazes you with flattering language the entire time so that chart go up.
People, the context-bearers, have experience and capabilities that machines might never understand encoded in our muscles and memory. I’m on record saying I despise nuance –and I do– but it’s more important than ever to be able to connect to our fellow humans over this nuance so our world is not paved over by contextless opinions from ill-informed robots. Empower and believe people over machines.