- Design
- Prospective shadow-mode diagnostic study
- Population
- 3,856 consecutive brain CT angiograms
- Primary outcome
- Sensitivity, specificity and incremental detection of AI vs radiologist
- Effect
- Sensitivity 0.846 vs 0.718; 55 AI-only true positives; NNE 70.1
An FDA-cleared aneurysm detection algorithm processed every brain CT angiogram at a US health system over six weeks, with radiologists blinded to its output. Discordant cases were adjudicated by a neuroradiologist.
Among 3,856 CTAs, 5.1% were positive. AI sensitivity was 0.846 (0.787 to 0.894) against 0.718 (0.649 to 0.779) for radiologists, with specificity about 0.985 for both. AI found 55 aneurysms radiologists had not reported; radiologists found 30 that AI missed. Most AI-only finds were under 3 mm, while radiologist-only finds were mostly 3 to 5 mm. The tool was most useful in inpatients (one extra aneurysm per 29 scans) and least useful in outpatients, where false alerts outnumbered true gains.
The tool and radiologist miss different aneurysms, which is the argument for using both. Many extra detections were small aneurysms of uncertain clinical significance, so the benefit to patients is less clear than the detection gain.
- Review AI-flagged regions systematically, especially the anterior communicating and posterior communicating segments.
- Report small aneurysms under 3 mm with size and location, and suggest neurovascular review rather than alarm.
- Expect more false alerts in outpatient screening populations.
- Audit concordance locally before relying on the tool.
Why it matters
It shows AI and radiologists miss different aneurysms, which argues for combined reading rather than replacement.
Don't overread it
Only discordant cases were adjudicated, so concordant misses were not counted; many extra finds were under 3 mm.
The statistics, in plain English
A number needed to examine of 70 means one extra true aneurysm for every 70 scans. A gain-to-pain ratio of 1.2 means slightly more true finds than false alerts overall; in outpatients it was 0.67, meaning more false alerts than gains.
Read the rest in the app
You have read your two free briefings this month. The app carries all 27 specialties, every morning, free — and this finding is waiting in it.
Scan to keep reading on your phone. No account needed to start.
Tomorrow morning, before your first patient
One edition a day for radiology, written by the desk, every claim tied to its paper. Six minutes.
Get the app — free