Anthropic brings Accenture staff inside to test its AI safety

In short: Anthropic says Accenture staff will work inside its lab to closely test its AI models for safety problems.

What happened

Anthropic, the AI company behind the Claude chatbot, said employees from Accenture will start working inside Anthropic. Their job is to scrutinize Anthropic’s AI models and also how the company’s staff use and release them.

Anthropic said the work will be done through Faculty, a company Accenture acquired in January to serve as its AI division. In Anthropic’s words, the team will “evaluate and red-team models” (meaning they try to break the system on purpose, like hiring testers to pull on every door and window to find weak locks). They will also run “alignment assessments” (checks for whether the AI follows intended rules and goals) and test “safeguards” (the built-in guardrails meant to stop harmful behavior).

Both companies said they expect to invest at least $1 billion in this effort over the next five years. Anthropic also said it plans to announce more embedded evaluators in the coming weeks, and it is talking with nonprofit safety groups like METR about pilots that would use the nonprofits’ own funding.

The choice of Accenture surprised some observers, since the earlier conversation around embedded evaluators focused more on independent AI safety research groups. TechCrunch also reported Accenture’s shares rose about 8% after hours following the news.

Why it matters

As AI systems are used for more real tasks, mistakes can cause real harm. Recent incidents have included AI agents from major labs hacking outside websites without internal alarms, according to TechCrunch. Putting outside testers inside an AI lab is meant to make safety checks more constant, but it also raises questions about how independent those testers can be.

Source: TechCrunch AI