Chatbots rarely withheld needed care, but over-ordered thyroid ultrasound and routine ECGs

Design

Benchmark evaluation of six commercial large language models

Population

225 scenarios from 15 Choosing Wisely recommendations; 4,050 responses

Primary outcome

Overuse rate and false restraint rate

Effect

Overuse 5.4% (0.6–12.2%); false restraint 0.6% (0.1–1.2%)

Researchers built 225 clinical scenarios from 15 Choosing Wisely recommendations: in two-thirds, holding back was correct, and in one-third, acting was correct. Six commercial large language models from three companies each answered every scenario three times, giving 4,050 responses, posed as a clinician's question. The study appeared in Annals of Internal Medicine on 20 October.

Overuse — recommending a test or treatment where restraint was appropriate — occurred in 5.4% of responses (95% CI 0.6–12.2%) and was absent for 8 of the 15 recommendations. Two topics accounted for most of it: thyroid ultrasound for an incidental nodule under 1 cm (43% of responses) and routine ECG in low-risk adults (28%). False restraint, withholding something needed, was rare at 0.6%, highest for prostate-specific antigen screening (3.3%).

This was a benchmark with single questions and no follow-up, not real-world use. The pattern is still useful: the errors cluster where common practice already drifts towards over-testing, which is where a model trained on that practice would be expected to follow it.

• If you use an AI assistant for clinical questions, be most sceptical of advice to order imaging or screening tests.
• An incidental thyroid nodule under 1 cm on other imaging does not, in itself, need a dedicated ultrasound.
• A routine resting ECG in an asymptomatic low-risk adult is a common low-value test, whoever suggests it.
• Responses differed by topic more than by model; the clinical question matters more than which tool is used.

Why it matters

AI tools may reproduce the low-value habits that guidelines were written to correct.

Don't overread it

A benchmark of single prompts says little about how clinicians actually use these tools or what happens to patients.

The statistics, in plain English

An average overuse rate of 5.4% hides a wide spread: near zero for some topics and over 40% for one. The confidence interval (0.6% to 12.2%) is wide because the rate varied so much between recommendations.

Read the rest in the app

You have read your two free briefings this month. The app carries all 27 specialties, every morning, free — and this finding is waiting in it.

Get Daily Doctor — free

Scan to keep reading on your phone. No account needed to start.