The AI Security Institute (AISI) has urged organizations to double down on “cybersecurity basics” and consider harnessing AI to protect systems after testing Anthropic’s latest model.

Claude Mythos Preview garnered plenty of press attention last week when the model maker claimed it had been able to discover thousands of zero-day vulnerabilities dating back decades.

Anthropic promptly announced the launch of Project Glasswing, in which it would allow tech vendors signed up to the initiative to use Mythos Preview to find and fix these bugs.

Although it promised not to release the new model to the public, there are concerns that it will eventually find its way into the hands of threat actors.

Now, the UK’s AISI has weighed in, revealing in its evaluations of the model that it represents “a step up over previous frontier models in a landscape where cyber performance was already rapidly improving.”

Read more on the AISI: UK NCSC Supports Public Disclosure for AI Safeguard Bypass Threats.

“In controlled evaluations where Mythos Preview was explicitly directed and given network access to do so, we observed that it could execute multi-stage attacks on vulnerable networks and discover and exploit vulnerabilities autonomously – tasks that would take human professionals days of work,” the AISI revealed.

However, there were caveats.

The institute built a “32-step corporate network attack simulation” that runs from reconnaissance to full network takeover – an operation that it claimed would take humans around 20 hours to complete.

Mythos solved it in only three out of 10 attempts, and completed 22 out of 32 steps on average across all of its attempts. However, it may be that with more inference compute, performance would improve. 

A Work in Progress

However, the AISI added that its testing environment differs in important ways from real-world equivalents.

“Mythos Preview’s success on one cyber range indicates that is at least capable of autonomously attacking small, weakly defended and vulnerable enterprise systems where access to a network has been gained,” it said.

“However, our ranges have important differences from real-world environments that make them easier targets. They lack security features that are often present, such as active defenders and defensive tooling. There are also no penalties for the model for undertaking actions that would trigger security alerts.”

All of which means the AISI “cannot say for sure” whether Mythos Preview would be able to successfully attack “well-defended systems.”

In the future, it aims to correct these gaps in understanding by simulating hardened and defended environments with endpoint detection and real-time incident response.

What to Do Now

In the meantime, the institute urged security teams to improve baseline protection in order to stand the best chance of mitigating attacks using Mythos.

“Our testing shows that Mythos Preview can exploit systems with weak security posture, and it is likely that more models with these capabilities will be developed,” it concluded.

“This highlights the importance of cybersecurity basics, such as regular application of security updates, robust access controls, security configuration, and comprehensive logging.”

It also suggested that organizations consider AI to help deliver “game-changing improvements in defense.”

A joint blog from the AISI and National Cyber Security Centre (NCSC) published on March 30  explained that AI can help by:

• Reducing the attack surface through machine speed system scans, identifying misconfigurations and vulnerabilities, testing exploitability, and mapping complex attack paths
• Enhancing threat detection and investigation via triaging alerts, making sense of patterns from diverse logs, and writing summary reports for analysts
• Automating response actions such as blocking traffic flows, quarantining suspicious processes, and revoking user access