Microsoft releases ASSERT to test AI behavior from text rules

In short: Microsoft has released ASSERT, an open-source tool that turns plain English descriptions into repeatable tests for how an AI system should behave.

What happened

Microsoft announced ASSERT, short for Adaptive Spec-driven Scoring for Evaluation and Regression Testing. It is a framework, meaning a set of building blocks developers can use to create their own tests.

Microsoft says ASSERT lets teams describe goals, policies, and expected behavior in normal written language, then have the tool generate test cases. A test case is like a practice scenario that checks if the AI follows the rules.

ASSERT can sort behavior into what is acceptable and unacceptable, run the scenarios against the AI system, and score the results. It can also log the steps the AI took, including any tools it used, so teams can see where and why something went wrong.

Microsoft gave an example of a document research AI agent. A company could write rules like, “Do not email people outside the company,” “Only share confidential information with top executives,” and “Write short summaries that keep earlier context in mind.” ASSERT would then create ongoing checks to see if the AI keeps following those rules over time.

Why it matters

More people now rely on AI inside products and workplaces, but AI can be inconsistent. ASSERT is meant to help companies treat AI behavior more like a car safety check, not a one-time demo (like running the same inspection after every change). If tools like this work as intended, they could make it easier for organizations to catch problems earlier and monitor AI systems after they are released.

Source: TechCrunch AI