An aneurysm-detection AI found 55 aneurysms radiologists missed on 3,856 CT angiograms

Design

Prospective shadow-mode diagnostic study with discordance adjudication

Population

3,856 consecutive brain CT angiograms at a US health system

Primary outcome

Sensitivity, specificity and incremental detection

Effect

Sensitivity AI 0.846 (95% CI 0.787 to 0.894) vs radiologist 0.718 (0.649 to 0.779); 55 AI-only true positives

A prospective shadow-mode study in the Journal of the American College of Radiology (15 September 2026) ran an FDA-cleared commercial algorithm on every brain CT angiogram over six weeks at a US health system, 3,856 studies, with radiologists unaware of its output. Disagreements were adjudicated by a neuroradiologist.

Agreement was 96.3%. The AI's sensitivity was 0.85 against 0.72 for radiologists, with similar specificity (about 0.99 for both). It found 55 aneurysms the radiologists did not, while radiologists found about 30 the AI missed. Most AI-only finds were under 3 mm; most radiologist-only finds were 3 to 5 mm. For every AI-only true positive there were about 0.8 false alarms, and 70 scans were needed per extra find. The yield was best in inpatients (one extra aneurysm per 29 scans) and poor in outpatients (one per 130, with more false alarms than finds).

Many extra aneurysms were tiny, and their clinical importance varies. The useful message is about deployment: the same tool can be worth it on the ward and a burden in the outpatient list.

• Use AI as a second reader on CT angiography, not a replacement; each missed the other's aneurysms.
• Expect most AI-only finds to be under 3 mm; agree with neurosurgery how these are reported and followed.
• Review yield and false alarms by setting before switching on AI for outpatient lists.
• Track the false-positive workload as part of any AI deployment.

Why it matters

It measures the real trade of AI deployment, extra finds against extra checks, setting by setting.

Don't overread it

Only disagreements were adjudicated, so true sensitivity may be overestimated, and many extra finds were too small to change management.

The statistics, in plain English

Sensitivity of 0.85 means the AI flagged about 85 of every 100 aneurysms present. Number needed to examine of 70 means 70 scans had to run through the AI to find one aneurysm the radiologist missed. A gain-to-pain ratio above 1 means more extra true finds than extra false alarms.

Read the rest in the app

You have read your two free briefings this month. The app carries all 27 specialties, every morning, free — and this finding is waiting in it.

Get Daily Doctor — free

Scan to keep reading on your phone. No account needed to start.