The Microsoft Detection and Response Team (DART) has issued advice on how organizations and their security teams should respond to the rising issue of AI-powered cyber threats.

“AI is amazing, it makes our job easier. “But the same AI that’s useful can be easily manipulated by threat actors, we’ve seen it in social engineering and in our day-do-day investigations," said Mary Asaolu, senior security researcher at Microsoft, during Infosecurity Europe on June 3.

In addition, while AI is being deployed within the enterprise to provide benefits to organizations and employees, if not managed correctly, AI code can introduce cybersecurity risks.

“AI really is the emergent angle,” said Meaghan Bradshaw, principal security researcher at Microsoft. “But AI code introduces another layer of risk. Nearly half of AI code contains flaws. Attackers can exploit it to compromise applications or data.”

This is not a theoretical concept: cyber criminals have already exploited AI tools as part of the attack chain, as demonstrated during Microsoft's Infosecurity Europe talk titled ‘Securing AI in the Age of Intelligent Threats’, which detailed a campaign dubbed ‘JustAskJacky’.

The JustAskJacky attack tricks users into downloading what looks like a legitimate AI assistant, but is in fact a backdoor which cybercriminals use to deliver malware.

The campaign combines this with professional-looking interfaces and valid digital signatures which make it harder for both users and security tools to distinguish it from legitimate software, thus allowing it to stay under the radar.

In fact, the malicious AI assistant was so good at avoiding detection that it was only discovered when Microsoft DART was brought into an organization to investigate a separate issue.

“They found this application was masquerading as an AI assistant to help day-to-day workflows,” Bradshaw added

At first glance, it appears to function normally; however, during installation, a backdoor written in Java is deployed, along with a persistence mechanism that creates a scheduled task running every four hours to maintain control and send telemetry.

The lesson to take from this, Bradshaw explained, is that organizations and users need to take a step back and think about what AI services they are installing and where they come from, because threat actors know that employees are looking for AI tools.

“Everyone is excited to leverage it to enhance the day-to-day. But on the other side, it often leads to users putting their guard down and not knowing what they are running. All it takes is one user to be convinced to gain that foothold,” she said.

“One of the most common recommendations we give customers is to take the time to assess nonstandard applications installed. If there is no business need, get rid of them. Because as much as it is useful for you, it is useful for threat actors too. Make sure you know what employees are using,” Bradshaw added.

Securing AI Augmented Employees

Like many cybersecurity challenges, one of the best ways to solve a problem - in this case, the cybersecurity risks around installing AI applications – is to get ahead of it.

Throughout the business, from board level to junior staff, employees should be informed about the potential risks around downloading unauthorized AI tools and be provided with information on how to safely adopt and deploy AI assistants.

“Provide a clear roadmap for safe adoption,” said Asaolu. “Make AI security a leadership priority, ensuing you have security reviews in place and AI is at a board discission level.”

“Ensure AI is used responsibility, make sure that good AI is the default behavior. And ensure security teams are equipped and coordinated for carrying out risk assessments and monitoring for unusual behavior,” she added.