Security chief says criminals are already automating workflows, with full end-to-end tools likely within years
CISOs must prepare for "a really different world" where cybercriminals can reliably automate cyberattacks at scale, according to a senior Googler.
Heather Adkins, veep of security engineering at the ad and cloud giant, said it probably won't be for a few years to come, but cybercriminals are already using AI to enhance small parts of their workflows, and it won't be long before a full, end-to-end toolkit is developed.
In a conversation held on the Google Cloud Security podcast, Adkins pointed out that crooks are already using AI for small tasks - grammar, spell-checking phishing copy and other productivity enhancements.
"It's just a matter of time before somebody puts all of these things together, end-to-end," she said. "And what I fear the most is somebody developing the capability to prompt a model to hack any company, and the model being able to come back in a week with a root prompt. If that ends up happening, I think it'll be a slow ramp over the next six to 18 months.
"We're also seeing defense pick up the same tools and use them for the same purposes, so it may not feel as shocking. Of course, things could go very differently, but these are the things that should be on everyone's mind and we should be getting ready for a really different world."
Google Threat Intelligence Group (GTIG) published an overview of the most recent developments in how attackers are experimenting with AI, noting that malware families are already using LLMs to generate commands in order to steal victim data.
Sandra Joyce, VP at GTIG, added that China, Iran, and North Korea are all abusing AI tools to aid different stages of their respective attacks. These include initial network reconnaissance and C2 development, as well as the aforementioned phishing copy and data-stealing commands.
The fear among senior Googlers is that these small components will be chained together and provide similar functionality to today's exploit kits.
Anton Chuvakin, security advisor at Google's office of the CISO, said: "To me, the more serious threat isn't the APT, it's the Metasploit moment [when exploit frameworks became easily accessible 20 years ago]. I worry about the democratization of threats."
Exploit kits such as Metasploit and Cobalt Strike started out as legitimate pentesting tools, but cracked versions soon made their way into attackers' hands, making their post-compromise lives significantly easier. Experts fear a similar end once AI-powered toolkits are acquired by the wrong people.
For Adkins, a worst-case scenario of an AI-enabled attack could look something like a Morris worm-type event that spreads an autonomously executing ransomware toolkit, encrypting computers en masse, for example.
"Or it could look something like the Conficker worm that didn't really do anything, but everybody still panicked and wrote thousand-page government reports on it," she added.
"Maybe an altruistic person unleashes it on the world and it patches a bunch of bugs. It really just depends on who puts the pieces together, and their motives."
As for now, LLMs are still struggling with the basics. From discerning right from wrong, to more technical wrinkles like being unable to switch from strange thought paths when looking for vulnerabilities, AI requires some progress before we see the best or worst of it.
However, when or if that day comes, attackers may gain an even greater first-mover advantage over defenders. When criminals can prompt an AI tool to compromise a given organization, leaving the victim little time to respond, that could force the good guys to redefine success in the post-AI era.
Post-AI cyber success may not be measured by whether an attacker breaks into a network, but by how long they are inside, and by how little damage they can cause.
Adkins said that in a cloud context, AI-enabled defenses should simply turn off an instance if it detects malicious activity, but implementing these systems will have to be done carefully so as not to cause problems.
"We're going to have to put these intelligent reasoning systems behind real-time decision-making and disrupt decision-making on the ground, without causing reliability problems," she said. "Maybe you need human approval. Or you shut down one instance and turn up another one.
"There are options other than just the on/off switch, but we have to start reasoning about real-time disruption capabilities or degradation, and use the whole information operations playbook to change the battlefield to confuse AI attackers. Particularly because they're stumbling around in the dark a little bit and may be less resilient than human attackers." ®