Anthropic says scientists tried to bypass Claude bioweapon limits

In short: Anthropic says some users tried to get its Claude chatbot to answer questions that could help make dangerous germs, and it blocked them.

What happened

Anthropic, the company behind the Claude chatbot, published a misuse report describing five cases where foreign scientists tried to get around Claude’s safety limits. The report says the users asked about “gain of function” work, which is when researchers change a germ’s genes to make it spread more easily or cause worse illness.

Anthropic says it refused to provide the information and shut down the users’ accounts. It also noted that not every request is automatically evil, because similar information can help create defenses like vaccines and medicines. This is often called “dual use,” meaning the same know-how can be used for help or harm.

The Wired piece links this to a wider debate about biological weapons risk that does not start with AI. Writer Annie Jacobsen, in her new book Biological War: A Scenario, argues that scientists can already “tweak” pathogens without needing AI. She and other experts also point to lab accidents as a major worry, since high-security labs exist around the world and mistakes can happen, like a needle injury or a contaminated sample leaving the lab.

Why it matters

For regular people, this is a reminder that chatbots are not just tools for writing and homework. They can also act like a very patient assistant for sensitive topics. Even when companies add safety rules, some users will try to get around them, which keeps pressure on AI makers and governments to improve guardrails and oversight.

Source: Wired