Nvidia said that the mission of the alliance is ‘to ensure defenders everywhere have open, frontier tools they can trust and control’.

Chipmaker Nvidia has formed an alliance with other companies in the technology space, in order to develop and share tools designed for enhanced AI safety and cybersecurity. 

The Open Secure AI Alliance’s founding members include Adobe, Crowdstrike, Hugging Face and Dell Technologies and follows a ​public letter, signed by a wide range of companies ​including OpenAI, advocating for open-weight AI models.

In a blogpost announcing the alliance, Nvidia said the coalition will work to “remediate and disclose vulnerabilities using open technologies”.

The creation of the coalition comes after AI and machine learning platform Hugging Face recently found itself the subject of a major cyber breach initiated by a rogue OpenAI agent, in which the company was unable to defend itself with US AI models due to guardrails being unable to distinguish between aggressor and defender.

In order to quell the attack, the platform turned instead to a self-hosted, open-weight Chinese model, which was not subject to the same restrictions.

“[The Hugging Face] incident showed a practical truth: when defenders cannot inspect, adapt and run advanced AI on their own infrastructure, their ability to respond is constrained at exactly the moment speed matters most,” said Nvidia. “Companies and countries need open frontier defensive tools and techniques so critical industries can build security systems across a multi-vendor ecosystem and avoid single points of failure.

“That is the mission of the Open Secure AI Alliance: to ensure defenders everywhere have open, frontier tools they can trust and control.”

Gene Moody, the field CTO at Action1 said the formation of the alliance is a “good step towards accountability”.

“Likewise it reflects an important recognition that the technology is advancing faster than many organisations can responsibly govern it,” he said.

“It would be a mistake however to believe those efforts meaningfully limit what determined threat actors can ultimately do, or what could one day be a simple mistake such as the Crowdstrike code update ‘accident’.”

He added: “The reality of the matter is that malicious actors have never depended on permission from anyone. Open-source models already exist by the thousands, many capable of running entirely on local hardware, disconnected from any cloud service or vendor oversight. 

“Once a model is operating in isolation, safety controls become just another layer of software. They can be modified, removed, retrained or replaced altogether. You cannot program a conscience into an artificial intelligence. You can only program behaviours, and behaviours are subject to manipulation by anyone with sufficient technical skill.”

Anthropic concerns

Anthropic’s CEO Dario Amodei recently aimed to dispel accusations that his company wants to ban Chinese open-weights models in the US, as a means of protecting its business. 

In a post on the Anthropic website yesterday (27 July), Amodei claimed that Anthropic does not advocate for a ban on open-weights models. He said that open-weights models without dangerous capabilities serve a public good that cost nothing beyond the computing power needed to run them and provide value to businesses, developers and researchers. 

He explained that his concerns lie in the use of more advanced AI by authoritarian governments and the wider risk that powerful AI models may be misused to carry out cyber or biological attacks.

Moody said: “We should be far more deliberate about the information we feed into all systems, not just AI. We should reconsider the authority we grant them, and the degree to which we depend on them. We simply do not possess the ability to secure these systems to the level that many people already trust them and that gap is growing.

“We have chosen convenience and novelty over safety. As AI capabilities accelerate, that choice carries unacceptable consequences. As long as valuable data, critical infrastructure and financial assets remain behind digital locks, someone will always be working to pick those locks. 

“AI does not eliminate that; it instills a false sense of trust that the tech will one day solve this problem, when in fact, we are the problem.”

With additional reporting by Colin Ryan

Don’t miss out on the knowledge you need to succeed. Sign up for the Daily Brief, Silicon Republic’s digest of need-to-know sci-tech news.