Models to predict checkpoint inhibitor pneumonitis were not ready for clinical use

Design

Systematic review and meta-analysis of prediction model studies

Population

12 models in lung cancer patients on checkpoint inhibitors (4,875 patients, 1,057 pneumonitis events)

Primary outcome

Validation C-statistic for checkpoint inhibitor pneumonitis

Effect

Pooled C-statistic 0.82 (95% CI 0.75 to 0.88), I2 90.7%, prediction interval 0.51 to 0.95

This systematic review of 12 models for predicting pneumonitis during checkpoint inhibitor treatment in lung cancer (4,875 patients, 1,057 events) pooled validation C-statistics. It was published in the Journal of Visualized Experiments, searching to August 2026.

The pooled C-statistic was 0.82 (95% CI 0.75 to 0.88) with very high heterogeneity (I2 = 90.7%), and the 95% prediction interval, which the authors identify as the decision-relevant quantity, was 0.51 to 0.95. Models validated only internally reached 0.76 to 0.83, and only two reported external validation (0.84 and 0.92). Imaging-based models scored higher than clinical models (0.85 vs 0.78) but the difference was not significant. Risk of bias was high in two, unclear in nine and low in one study, and the certainty of evidence was low.

The averages look good; the range is unusable. A model that might perform no better than chance in a new centre cannot guide treatment, so vigilance for symptoms remains the standard approach.

• Pooled validation C-statistic was 0.82, but the 95% prediction interval was 0.51 to 0.95.
• Only 2 of 12 models had external validation.
• Risk of bias was high or unclear in 11 of 12 studies, and certainty of evidence was low.
• Imaging models were not significantly better than clinical models (0.85 vs 0.78).
• Do not use these models to withhold or stop immunotherapy.

Why it matters

Published accuracy for such models is often high, and this review shows how little of it survives external testing.

Don't overread it

The review pools heterogeneous models of lung cancer and says nothing about models for other tumour types.

The statistics, in plain English

A prediction interval shows the range of performance a new model like these is likely to have in a new setting. Here it runs from chance level (0.51) to excellent (0.95), so the average figure hides how unreliable results may be. I2 above 90% means the studies differ greatly among themselves.

Read the rest in the app

You have read your two free briefings this month. The app carries all 27 specialties, every morning, free — and this finding is waiting in it.

Get Daily Doctor — free

Scan to keep reading on your phone. No account needed to start.