- Design
- Systematic review with random-effects meta-analysis and narrative synthesis
- Population
- 27 studies of AI, machine learning and tele-triage in emergency care
- Primary outcome
- Discrimination for critical illness or deterioration
- Effect
- Pooled AUC 0.874 (0.845–0.898), I² 94%
A systematic review in the Journal of Medical Internet Research included 27 comparative studies of AI, machine learning, natural language processing and tele-triage at first clinical contact in emergency and acute care.
In four adult emergency department studies predicting critical illness or deterioration, AI-assisted triage had a pooled AUC of 0.87, with very high heterogeneity. Where within-study comparisons were available, AI models outperformed conventional triage by 0.07 to 0.12 AUC units. Tele-triage was associated with faster assessment and fewer patients leaving without being seen. Calibration reporting, external validation and paediatric evidence were all thin.
Good discrimination is not the same as better decisions at the bedside. The authors' modelling suggested fewer missed critical cases at the cost of more false escalations, and no study showed better patient outcomes. Any department considering such a tool should validate it locally.
- Do not replace structured triage with an AI tool on current evidence.
- Ask for local validation and calibration data before adopting any AI triage product.
- Expect more false-positive escalations as the trade-off for fewer missed critical cases.
- Check performance in children and other subgroups separately.
Why it matters
Vendors are selling triage AI on AUC figures that say little about real decisions.
Don't overread it
An AUC of 0.87 does not show the tool improves outcomes or outperforms clinicians at the thresholds that matter.
The statistics, in plain English
An AUC of 0.87 means the model ranks a randomly chosen critically ill patient above a non-critical one 87% of the time. It says nothing about how many patients are wrongly escalated or missed at a given cut-off.
Read the rest in the app
You have read your two free briefings this month. The app carries all 27 specialties, every morning, free — and this finding is waiting in it.
Scan to keep reading on your phone. No account needed to start.
Tomorrow morning, before your first patient
One edition a day for emergency & critical care, written by the desk, every claim tied to its paper. Six minutes.
Get the app — free