Dwarkesh Patel urges AI labs to share evidence behind bold AI claims

In short: A podcaster says AI labs should publicly back up big claims after he heard a reported nine month timeline for AI beating human AI researchers.

What happened

Dwarkesh Patel said an unnamed researcher at Anthropic told him, “I expect A.I.s to beat humans at A.I. research within nine months.” Patel discussed the comment on his show “Interesting Times,” and argued that if labs believe this, they should share the evidence behind it.

Patel’s concern is not just the prediction. It is that the researcher reportedly would not explain what data or tests led to that timeline. Without details, outside experts cannot evaluate whether “beat humans” means writing better code, coming up with new ideas, running experiments faster, or something else.

A key caveat is that this is a reported forecast, not a verified result. Patel did not name the researcher, and he did not identify which AI model was involved, how it was measured, or what “beat humans” was supposed to mean.

Anthropic does publish some related information through a public “AI-work index.” That index tracks how much Anthropic’s Claude is involved in the company’s research and development work, and how much humans still supervise it. It is more like a workplace logbook than a head to head skills test, so it does not directly show that AI is outperforming human researchers.

Why it matters

If AI can help build better AI, progress could speed up quickly, like a factory that starts building machines that improve the factory. Big claims like this can influence policy, investment, and safety planning, so Patel argues the public should be able to inspect the evidence.

Source: NYTimes