AI leaders argue over who should control releases and safety rules

In short: A growing public debate is forming around a simple question, should AI companies police themselves, or should governments set and enforce safety rules.

What's going on

Some AI leaders are warning that AI could become dangerous if it is built too fast and without strong safeguards. Anthropic CEO Dario Amodei has called for slowing down parts of AI development and adding more protections. Reporting has also highlighted worries about “AI agents,” which are AI tools that can take actions on their own (like a digital assistant that does not just answer questions, but also clicks buttons and completes tasks).

At the same time, executives disagree on who should decide when new AI models are released. Nvidia CEO Jensen Huang has argued that AI companies should make those calls, not governments. Meta CEO Mark Zuckerberg has said market pressure can encourage safety, meaning companies will avoid harmful releases because it could hurt their business.

Critics say this leaves too much power with a small group of companies that profit from moving fast. Supporters say companies have the expertise and face legal and financial risk if something goes wrong.

What to watch

A key tension is that some leaders warn about extreme risks while still building and shipping more capable AI. Experts also do not agree on whether a worst-case “loss of control” scenario is likely, or when it could happen. What is clearer, and more immediate, are concerns like misuse, job disruption, privacy, environmental costs from data centers, and too much control concentrated in a few firms. Expect more calls for independent testing and clearer public rules, not just voluntary promises.

Source: NYTimes