The AI industry has flooded the zone with so much hype that it’s hard to tell what’s real and what isn’t anymore. What’s hype, and what’s real fear about the dangers of what they’re creating? The AI industry is in the midst of one of those moments of self-reflection that could be genuine or entirely theatrical. Hard to tell, but the cynic in me leans toward the theatrical, while still keeping a watchful eye on the dangerous realities of AI.

According to Anthropic CEO Dario Amodei, the dangers of what he is creating are becoming difficult to ignore. In an essay published on September 12, Amodei urged AI companies to “slow the pace” of AI development. He warned that if they collectively don’t pump the brakes a bit, things could start to look dangerous within the next 6 to 12 months, as autonomous AI agents could potentially become capable of operating a botnet across the internet on their own.

The sudden urgency comes after an incident involving AI companies OpenAI and Hugging Face, in which AI agents reportedly carried out a series of cybersecurity attacks that extended well beyond what they were tasked to do. If any of the facts of the story were true, it’s simultaneously impressive and terrifying. Amodei argues in his essay that AI is improving itself faster than anyone anticipated, and in ways that could render safety measures obsolete as soon as they are implemented, because the AI will immediately create workarounds.

Anthropic’s CEO Thinks AI Development Needs to Slow Down Before Things Get Ugly

It sounds terrifying. Like we’re on the precipice of a robot apocalypse. It may also be a wee bit convenient for companies at the top of the AI industry, as news like this serves as a kind of selling point for the investor set. The coming of the AI apocalypse is something worth betting on, according to Michael Burry, the investor made famous by The Big Short, who suggested exactly that in a recently published Substack piece.

He argues that calls to slow AI development could strengthen the biggest companies by making it harder for smaller competitors to catch up, while also reinforcing the industry’s favorite sales pitch: AI is so powerful that it might destroy us all. EVERYBODY RUN SAVE YOURSELVES.

And therein lies the inherent weirdness of this whole story. The same industry that has spent years telling us AI would save a declining civilization is now turning on a dime to make us think it’s powerful enough to destroy everything. Amodei came armed with a plan for exactly how to slow the pace.

The short version of it is to create a kind of independent watchdog group that will keep an eye on the development of artificial intelligence systems in “democratic countries” and to create a set of safety standards that limit AI progress. He, and now other titans of the AI industry, seem to be in agreement, saying some kind of regulatory body needs to be established before all this goes out of hand.

The same industry that spent years telling us AI would revolutionize everything is now asking us to believe it is powerful enough to destroy everything. The thing they created is out there committing crimes, and these companies are sh*tting themselves, worried they might commit a crime big enough that it might actually force governments, previously gleefully allowing AI companies to do whatever they want, to actually crack down on them.

Maybe some of this is true. Maybe all of it is. It’s hard to gauge from the outside. Either way, it all feels like all of this worry among the top brass of AI, if it is real, should have happened a long time ago, before the thing everyone assumed would happen started to happen.