Anthropic says it stopped several instances of foreign scientists using its AI agents to create possible biological weapons, the New York Times reports.
The alarming news was revealed in a lengthy report released by the Claude chatbot-maker on Thursday. Notably, it comes amid heightened AI safety concerns after a former Anthropic employee accused the company of irresponsibly building powerful AI systems despite knowing they’re destructive.
According to the report, scientists in several countries where Anthropic has blocked access to its models subverted these controls and tried to “obfuscate the purpose of their research to evade our safeguards.”
In one case in May, a scientist asked Claude to help write an application to receive a state-sponsored grant for an eyebrow-raising project: to engineer more harmful mutations of the mosquito-borne chikungunya virus, which can cause fever and debilitating joint pain. The project, Anthropic believes, was intended to be carried out at a military research institute.
“What we don’t know is if the research was meant to be weaponized,” Jacon Klein, Anthropic’s head of threat intelligence, told the NYT. “But a military institution doing gain-of-function research is concerning,” he added, referring to research that seeks to enhance or add to the biological capabilities of organisms and pathogens.
The company documented four other similar cases where scientists evaded its guardrails to pursue shady biological research. It banned all of the offending accounts. But, as with the chikungunya virus case, Anthropic couldn’t definitively determine if these inquiries were part of legitimate research or an attempt to engineer bioweapons. It nonetheless chose to err on the side of caution.
“You are not seeing someone in a comic book kind of way say, ‘Hey, I want to build a biological weapon to kill everybody,'” Klein told the NYT. “It’s an incredibly nuanced situation.”
For years now, the AI industry has warned of the risk of AI being used to create bioweapons, including Anthropic’s CEO Dario Amodei. In a lengthy essay published earlier this year, Amodei vowed that the company was constantly improving its guardrails to prevent models from assisting with bioweapon research and opined that legislation and international cooperation was necessary to mitigate the risks.
Still, while Anthropic claims to have thwarted these misuses of its models, the news casts doubt on whether it and other AI companies can guarantee the responsible use of its tech. By the company’s own admission, its safeguards failed, potentially leaving the door wide open for other bad actors in the future.
In short, if these state actors were indeed looking to craft bioweapons, it’s a dramatic escalation, suggesting we’re edging closer to a catastrophe. That’s of course if general purpose models like Anthropic’s are capable enough to meaningfully assist these dangerous efforts; amid the heightened safety debate, the company has faced accusations of fear-mongering about its own tech as a marketing tactic.
Some experts are of the mind that the threat is very much already real.
Andrew Weber, a senior fellow on the Council on Strategic Risks who reviewed Anthropic’s report ahead of its release, told the NYT that its findings were “chilling examples of state-sponsored biological weapons developers tapping into the rapidly advancing capabilities” of frontier AI models.
More on AI: Anthropic Is Building a Huge Surveillance System to Spy on Anti-AI Activists and Predict Their Activities