Studies find two AI health tools match or beat doctors in simulations

FORMAT B - Trend

In short: Special medical AI tools are showing stronger results than doctors in controlled tests, but researchers say they are not ready for routine clinic use.

What's going on

Two studies published in Nature tested “health models”, which are AI systems trained to handle medical information and suggest diagnoses and care plans. In both studies, the AI tools matched or surpassed doctors on certain tasks.

One tool, called Mira, was developed by researchers in Germany. It uses information from electronic health records, meaning the digital files hospitals keep about patients. In tests based on more than 500 emergency department cases, Mira reached 87.1% diagnostic accuracy across eight conditions, compared with 78.1% for a panel of six doctors.

The other tool, Google’s Amie, was tested against 21 primary care doctors using 100 multi-visit case scenarios based on UK medical guidelines and drug recommendations. Actors role-played patients and Amie responded using Google’s Gemini model. Researchers said Amie produced investigation and treatment plans that lined up more closely with the guidelines than the doctors’ plans, and it did better on medication decisions in difficult cases.

Researchers and outside experts stressed a key limit: these were controlled simulations, not real clinic visits. The “patients” gave information in a more organized way than many real people do, and real healthcare is often messy and incomplete.

What to watch

The big question is whether these tools can stay safe and reliable in real-world settings, where time is short and information is missing. Some experts also questioned whether Amie’s advantage comes from medical design choices, or simply from fast overall improvements in general AI. For now, researchers compare these systems to an airplane autopilot, helpful for routine tasks, but with doctors still responsible for final decisions.

Source: Financial Times