Microsoft has unveiled two new open-source tools called RAMPART and Clarity to assist developers in better testing the security of artificial intelligence (AI) agents.
RAMPART, short for Risk Assessment and Measurement Platform for Agentic Red Teaming, functions as a Pytest-native safety and security testing framework for writing and running safety and security tests for AI agents, covering both adversarial and benign issues, as well as various harm categories.
Users can write test cases to attack or probe an AI agent to explore possible safety violations like cross-prompt injections, where untrusted data reaches an AI system indirectly via a data source (e.g., email, file, or a web page) processed by it, or unintended behavioral regressions and data exfiltration.
RAMPART then evaluates the outcome of those tests and reports the results. All it needs is an adapter that connects an agent to the test suite. The tool builds on PyRIT (short for Python Risk Identification Tool), which Microsoft released more than two years ago as a way to test AI systems.
Clarity, on the other hand, has been described by the tech giant as a "structured sounding board" to help developers arrive at the right approach even before writing a single line of code. It's an "AI thinking partner that pushes back," guiding them through problem clarification, solution exploration, failure analysis, and decision tracking.
In publicly releasing these tools, Microsoft said the idea is to address why certain decisions are incorporated at an early stage of software development so that any potential issue - for example, an agent's access to a tool - is addressed well before the system is built.
"We wanted to give product managers and engineers a way to pressure-test their assumptions at the start of a project, when changing course is cheap and the right conversation can save months of rework," Ram Shankar Siva Kumar, a Data Cowboy and founder of Microsoft's AI Red Team, said in a blog shared with The Hacker News.
Microsoft noted that a secondary motivation behind investing in these tools is to make incidents reproducible and mitigations verifiable and scale the learnings from red teaming exercises by turning them into runnable engineering assets.
"Where PyRIT is optimized for black-box discovery by security researchers after the system is built, RAMPART is built for engineers as the system is being built," Siva Kumar added. "Clarity helps teams clarify design intent and capture assumptions. Together, these approaches move AI safety from a one-time review to a set of living artifacts that developers can use throughout the lifecycle."
Found this article interesting? Follow us on Google News, Twitter and LinkedIn to read more exclusive content we post.