Reports warn many AI claims are ahead of strong evidence

In short: More reports and studies are warning that many “AI-powered” promises are being made before there is strong, independent proof that they work.

What's going on

Several researchers and news outlets say the benefits of many AI tools are being oversold. The AI Now Institute argues that AI’s benefits are often “overstated and underproven,” while the evidence of harms is stronger. It also points out that generative AI (tools that write or create images) can produce “hallucinations,” which is when a system makes something up and states it like a fact.

In education, Inside Higher Ed reports a growing research gap. Experts say there is still not enough independent evidence that AI tools lead to big improvements in learning. Some studies that claim large gains may use weak methods, which makes the results less trustworthy.

In healthcare, the gap between promotion and proof looks even bigger. One analysis in PLOS Digital Health found that 1,357 AI or machine learning medical devices had received clearance or approval from the U.S. Food and Drug Administration. Only 3 had been evaluated for patient-centered outcomes, like whether people lived longer or avoided returning to the hospital.

Drug discovery shows similar limits. A review of clinical trials reported that few AI-discovered drugs have reached human testing, and none have been approved for patients so far.

What to watch

AI is still showing real promise in areas like disease detection and biomedical research, but many advances need prospective validation, meaning they must prove themselves in real-world tests that track patient or student outcomes over time (like a road test, not just a lab demo). For now, experts say people should treat bold “AI-powered” claims as a starting point to be checked, not as proof.

Source: NYTimes