NYC Council presses AI companies for safety promises in hearing

In short: New York City Council members questioned major AI companies about worst case risks and got few concrete promises in return.

What happened

On October 5, 2026, the New York City Council held a hearing on advanced artificial intelligence. Representatives from OpenAI, Anthropic, Google, and Meta testified. Former industry employees also spoke and urged stronger oversight.

Lawmakers asked how likely a catastrophic AI incident might be, meaning a failure so serious it causes widespread harm. None of the companies put a number on that risk. An OpenAI representative said they did not know, and added that even a small chance would be unacceptable. A Google representative said there is not yet a rigorous way to calculate a probability.

Council members also pushed for clear commitments about safety checks. They asked whether a model would be held back if it failed an independent safety test, and whether a dangerous system could reliably be stopped by people, like a kill switch (an off button meant to shut something down fast). The companies described internal review processes, but did not broadly promise they would not release a model that failed outside testing. They also did not give a clear general assurance about shutoff controls or who would be legally responsible if serious harm occurred.

Why it matters

New York City is considering rules that could affect AI systems used by city agencies, contractors, or companies selling AI in the city. Proposals include third-party validation (outside checking), a required human override, whistleblower incentives, faster incident reporting, and possibly allowing harmed people to sue. The hearing showed a gap between general statements about safety and the specific guarantees lawmakers are looking for.

Source: NYTimes