Anthropic circulates draft AI safety bill with a 50-death risk trigger

In short: Anthropic is sharing a draft AI safety bill that could pause the release of an AI model in extreme risk cases, but only if a judge approves it.

What happened

Anthropic, an AI company, has circulated a draft bill that it hopes the US Congress will consider, according to Politico and reported by TechCrunch. The bill lays out situations where an AI model could be blocked from being released to the public.

The draft uses specific thresholds. It points to a “foreseeable, imminent risk” of more than 50 deaths, or more than $1 billion in property damage. The bill says the danger must be linked to things like creating a biological weapon, the AI system getting around its maker’s controls, or a cyberattack.

Under the proposal, “independent evaluators” would assess whether a model meets that catastrophic risk bar. Think of them like outside inspectors checking a building before people can move in. These evaluators would be registered with the US Department of Commerce, and they could be paid by AI labs as long as the payment is not tied to the outcome.

If evaluators believe a model poses imminent catastrophic risk, they would have 72 hours to notify the Commerce Department. The department would then need to ask a judge in Washington, D.C. to halt the release. The draft would also stop federal agencies from blocking a model on their own without a judge’s permission.

Why it matters

This proposal shows how one major AI company wants safety rules to work, and it sets a high bar for when the government can step in. For the public, the key question is who gets to hit the pause button when a powerful system seems dangerous, and how quickly that can happen.

Source: TechCrunch AI