California Governor Gavin Newsom proposes an AI kill switch plan

In short: California Gov. Gavin Newsom signed an executive order that could lead to a required “kill switch” for the most powerful AI systems.

What happened

California Governor Gavin Newsom issued a new executive order on Friday aimed at tightening AI oversight in the state. An executive order is a direction from the governor to state agencies on what work to start.

The order tells California to convene a group of experts and deliver recommendations within two months. The goal is to suggest updates to state law that would strengthen AI safety rules.

One idea the group must consider is an AI “kill switch,” meaning a proven way to quickly shut down a powerful AI model if something goes wrong (like an emergency stop button on a machine). The order also asks the group to look at requiring AI companies to host independent verification teams on site for regular audits, which are structured checkups of safety practices.

It also calls for stronger standards for transparency reports and risk assessments, with review by independent auditors. Newsom’s order says companies should be required to report “loss-of-control incidents” as critical safety incidents. These are cases where an AI system behaves in an unexpected or unsafe way, and the company cannot reliably stop or contain it.

The order also directs a state agency to speed up work on two recent California laws. Those laws set up a framework for independent verifiers to assess AI safety and a state registry of AI auditors.

Why it matters

If California moves forward, companies building advanced AI may have to meet tougher safety checks to do business in the state. Because many AI companies operate in California, state rules can sometimes influence what becomes common practice across the US.

Source: The Verge AI