Nilay Patel:

[E]veryone in tech understands how much regular people dislike AI. What I think they’re missing is why. They think this is a marketing problem.

[S]oftware brain [is] see[ing] the whole world as a series of databases that can be controlled with the structured language of software code.

Once you start seeing the world as a bunch of databases, it’s a small jump to feeling like you can control everything if you can just control the data.

It turns out software brain has a limit — the government isn’t software. People aren’t computers, and they don’t live in automatable loops that can be neatly captured in databases.

At some point, the database stops matching reality. At that point, we usually end up tweaking the database, not the world. But the AI industry has fully lost sight of this, because AI thrives on data. It’s just software, after all. And so the ask is for more and more of us to conform our lives to the database, not the other way around.

This intoxicating similarity between law and code trips people up all the time. People are constantly trying to issue commands to society at large like it’s a computer that will obey instructions.

But law isn’t actually code, and society and courts aren’t computers. […] You simply cannot take the facts of a case, the law as written, and predict the outcome of that case with any real certainty, even though the formality of the legal system makes people think it works like a computer — that it’s predictable.

But at the end of the day, it’s actually ambiguity that’s at the very heart of our legal system. It’s ambiguity that makes lawyers lawyers. Honestly, it’s ambiguity that makes people hate lawyers because it’s always possible to argue the other side, and it’s always possible to find the gray area in the law.

There’s real value in introducing AI to business, because so much of modern business is already software: collecting data, analyzing it, and taking action on it over and over again in a loop. Businesses also control their data, and they can demand that all their databases work together.

Most people are not collecting data about every single thing that they do. And if they’re collecting any at all, it’s stored across lots of different systems […] Those systems don’t talk to each other and maybe they never will, because there’s no reason for them to. Asking people to connect them all freaks them out.

[I]t is a failure when you ask people to adapt to computers. Computers should adapt to people. Asking people to make themselves more legible to software — to turn themselves into a database — is a doomed idea.

Not everything about our lives can be measured and automated and optimized, and it shouldn’t be.

And so the tech industry is rushing forward to put AI everywhere at enormous cost — energy, emissions, manufacturing capacity, the ability to buy RAM — and locked into the narrow framework of software brain without realizing they are also asking people to be fundamentally less human. They then sit around wondering why everyone hates them.