Anthropic safety lead says AI has over 10% chance to kill all humans

In short: An Anthropic AI safety lead said he thinks there is more than a 10 percent chance that AI could kill all humans within the next decade.

What happened

Evan Hubinger, who leads one of Anthropic’s AI safety teams, posted on X that he worries about “self-improving AI” and said it is happening faster than expected. He added that he agrees with a former colleague’s warnings. Hubinger said he personally estimates the chances of AI killing all humans are greater than one in 10 within the next decade.

This came just hours after Jacob Coxon, a researcher who helped train AI systems at Anthropic, said he quit the company over what he called a lax approach to safety. Coxon also said AI companies are “gambling with our lives” by racing to build “superhuman systems” they cannot control.

Both posts focus on the idea of “self-improving” AI, meaning an AI system that can help make the next version of itself, which then helps make an even stronger version, and so on (like a copy machine that can upgrade itself each time it prints a new copy). Some AI insiders worry this could become a runaway loop where people can no longer steer what the system does.

Hubinger also said Anthropic does “not yet have a plan” to ensure very advanced AI stays safe and matches human values, and that the company is “not clearly on track” to develop one.

Why it matters

These comments show that some people working inside major AI labs think worst case outcomes are not just science fiction. If they are right, it raises questions about how fast companies should build more powerful AI, and what rules and independent checks should exist before these systems are widely used.

Source: The Verge AI