Suicide prediction from routine health checks: good for attempts, weak for deaths

Design

Prognostic machine-learning study with 70/30 train–test split

Population

668,684 US Army soldiers; 1,932,243 periodic health assessments, 2015–2019

Primary outcome

Suicide and suicide attempt within 24 months

Effect

AUROC 0.72 for suicide, 0.81 for attempts; top 5% captured 18.6% of suicides

Investigators built machine-learning models from administrative data available at US Army periodic health assessments, aiming to pick soldiers for a remote group skills programme based on dialectical behaviour therapy. The data covered 1,932,243 assessments from 668,684 soldiers between 2015 and 2019, with outcomes followed over 24 months.

Over that window, suicide occurred at 66.7 per 100,000 and suicide attempts at 704.1 per 100,000. In the held-out test sample, the model discriminated attempts well (AUROC 0.81) and suicide less well (AUROC 0.72). Risk of suicide was meaningfully raised only in the top 5% of predicted scores, which contained 18.6% of all suicides; for attempts, the top 10% contained 46.5%. The two predicted risks correlated only weakly (r = 0.29).

The central point is that the people who die by suicide and the people who attempt it are not the same group, and a model tuned for one does not find the other. Even a well-calibrated model leaves more than four in five suicides outside its high-risk tier.

For clinicians, this supports using such tools to target low-cost, low-harm interventions at a population level, not to decide who is safe. The study is in a young, mostly male military population and the model is not transferable as it stands.

• A high-risk flag from an algorithm should prompt an offer of support, not replace a clinical assessment.
• A low predicted score does not mean low risk: most suicides in this study occurred outside the top-risk group.
• Ask about suicidal thoughts and recent attempts directly, whatever any screening tool reports.
• Treat suicide and suicide attempt as related but distinct outcomes with different risk profiles.
• Population models suit offering brief, low-harm interventions widely; they are not a basis for withholding care.

Why it matters

Even a large, well-calibrated model missed most suicides, which is the strongest argument yet against using risk scores to triage people out of care.

Don't overread it

This was a young, mostly male military population; the model's performance elsewhere is unknown.

The statistics, in plain English

An AUROC of 0.72 for suicide means the model ranks someone who died above someone who did not about 72% of the time, which is modest. Sensitivity of 18.6% in the top 5% means roughly four in five suicides happened among people the model did not flag as high risk.

Read the rest in the app

You have read your two free briefings this month. The app carries all 27 specialties, every morning, free — and this finding is waiting in it.

Get Daily Doctor — free

Scan to keep reading on your phone. No account needed to start.