Most AI safety testing still happens inside the walls of individual organizations. That has resulted in a fundamental disconnect: many of the highest-risk failure modes in modern AI systems require deep domain expertise, multilingual context, or regional understanding that no single internal team can fully replicate on its own.
As frontier models become more capable, the attack surface expands with them. AI red teaming is no longer just about prompt injection or content safety edge cases. It increasingly involves security operations, misuse scenarios, multilingual harms, alignment failures, and domain-specific abuse patterns that can vary significantly across geographies and languages.
Microsoft’s AI Red Team has observed that meaningful testing of advanced AI systems- and models similarly requires broader participation from researchers and practitioners who operate outside traditional corporate security boundaries. To address that gap, today we are announcing the External Red Team Alliance (EXTRA), a formalized global extension of Microsoft’s AI Red Team designed to support and encourage external expertise to advance AI safety and security testing. We are proud to share we are funding the development of new AI safety assessments on six continents through unrestricted gifts.
EXTRA is a two-part initiative focused on expanding AI safety research and strengthening external collaboration.
The first component supports a global academic network focused on advancing AI safety and security research. Microsoft’s AI Red Team has provided unrestricted gifts to 18 university labs spanning six continents. The goal is intentionally broad: support researchers who are already investigating difficult, unresolved questions in AI safety and help them continue pushing that work forward independently.
Some of the supporting institutions include:
“Academic research is critical to understanding the cyber security landscape and finding solutions that work for all of society – and partnerships like this with industry are essential to delivering on that promise. Through partnerships, civil society and public institutions researchers gain access to frontier technology to understand how models work and bring their expertise to the task of determining risk and developing more effective countermeasures for the benefit of society as a whole.”
Nicolas Papernot, professor, University of Toronto.
The second component of EXTRA focuses on operational collaboration. Microsoft is building a distributed network of specialists who can participate directly in red teaming highly specialized areas where deeper expertise is required. That includes researchers, practitioners, and regional experts who understand specific attack classes, languages, cultural contexts, or technical domains that internal teams may not fully cover alone.
Beyond expanding participation, EXTRA is also intended to help advance the science of AI safety evaluation. By bringing together academic researchers, security practitioners, and domain experts from around the world, the initiative aims to contribute to the development of more robust methodologies and testing practices for increasingly capable AI systems. Today’s cybersecurity ecosystem depends on coordinated vulnerability research, responsible disclosure programs, academic inquiry, and global communities of independent security researchers who routinely identify risks that vendors alone would not find. Likewise, advancing AI safety will benefit from ongoing contributions from experts across institutions, disciplines, and geographies to identify emerging threats, strengthen safeguards, and improve evaluation practices.
“As frontier model capabilities advance, they create new risk opportunities, particularly in low-resource settings. Partnerships, such as this EXTRA, bring greater attention to the study of the local risk landscape and can enable broader impact of the work carried out around the globe in smaller academic settings.”
Balaraman Ravindran, head of the Robert Bosch Centre for Data Science and Artificial Intelligence (RBC-DSAI) at IIT Madras.
The research areas funded through EXTRA reflect several of the emerging areas Microsoft’s AI Red Team continues to encounter when evaluating advanced AI systems.
Some universities are examining the cybersecurity implications of AI systems themselves — including how models can be attacked, manipulated, or abused in operational environments. Other labs are exploring the inverse problem: how AI systems can assist defenders and improve cyber operations.
The structure of the program is intentional. The funding is unrestricted because the objective is not to direct research outcomes toward product requirements or predefined deliverables. The goal is to strengthen independent safety research capacity globally and create stronger long-term collaboration between academia and operational AI security teams.
“Managing frontier AI risk requires more than internal safeguards. It requires continuous engagement with experts who understand how these systems behave across different technical, linguistic, and cultural contexts. EXTRA reflects Microsoft’s broader Frontier Governance Framework approach: combining rigorous internal governance with external support and collaboration to better identify, assess, and mitigate emerging risks as AI capabilities advance. By supporting independent research and building stronger connections with universities and specialists around the world, we are helping strengthen the broader ecosystem needed for trustworthy and secure AI development,” says Natasha Crampton, Chief Responsible AI Officer, Microsoft
Governments too are increasingly focused on understanding the capabilities and security implications of frontier AI systems to strengthen resilience. But just as coordinated international research helped unlock the benefits of previous technological revolutions, diverse expertise from researchers and practitioners around the world is essential to identify emerging threats, improve defenses, and build greater confidence in AI systems.
“Frontier AI is already shaping the future of both cybersecurity and national security. Understanding how these systems can be misused, and identifying risks before they become real-world threats, requires expertise that spans institutions, disciplines, and borders.” says Mike Yeh, VP & Deputy General Counsel, Customer Security and Trust, Microsoft
AI red teaming is becoming more interdisciplinary, multilingual, and globally distributed. The expertise needed to identify meaningful failure modes increasingly lives across universities, independent research communities, and regional specialists.
EXTRA reflects a practical shift in how AI security testing must operate going forward. External expertise is no longer supplemental to red teaming; in many cases, it is essential.
Microsoft would like to thank these people for their important contributions with this project: Steph Ballard, Blake Bullwinkel, Nicholas Butts, Janelle Bryant, Kaja Ciglic, Hector de Rivoire, Eugenia Kim, Amanda Minnich, Shujaat Mirza, Jingxia Ni, Saphir Qi, Giorgio Severi, Hilary Solan, Hiwot Tesfaye, Sam Vaughan, Marguerita Wicklander, and the many teams at these schools around the world who helped coordinate
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。