Nodule-matching AI failed mostly on pleural plaques, which is the useful part

Design

Retrospective standalone evaluation of a commercial AI pulmonary nodule system with fully automated longitudinal matching and expert review of failures

Population

All 361 UK Lung Cancer Screening trial participants who underwent a 3-month follow-up low-dose CT; 181 had 378 baseline nodules of 100 mm3 or more

Primary outcome

Successful automated matching of persisting baseline nodules at follow-up

Effect

83.5% of 339 persisting nodules matched (95% CI 79.2-87.1); 91.8% with a single nodule, 72.8% with more than five; 91.1% of failures were non-nodular, leaving 1.5% needing manual correction

Automated volume doubling time in lung cancer screening depends on matching the same nodule between two scans, and matching is the unglamorous step where automation usually breaks. This study tested a commercial pulmonary AI on all 361 UK Lung Cancer Screening trial participants who had a three-month follow-up low-dose CT, using an updated volume threshold of 100 mm3 for the solid component per the NELSON 2.0 and EUPS protocols.

The design deliberately removed the usual safety net: every AI-detected baseline candidate nodule of 100 mm3 or more went to automated matching with no manual pre-selection. The AI found 181 participants with 378 baseline nodules at or above threshold; 39 had resolved by follow-up. It matched 83.5% of the 339 persisting nodules (95% CI 79.2-87.1). Performance depended heavily on burden: 91.8% where there was a single baseline nodule, which described 59.7% of the cohort, and 72.8% in the 6.6% of participants with more than five nodules.

Expert review of the 56 unmatched findings is what turns a mediocre-looking number into a good one. 91.1% of them were not nodules at all - mostly pleural plaques, 46.4% of the failures. Only five genuinely discrete solid nodules, 1.5% of persisting findings, actually needed manual correction.

So the headline matching rate understates the system, and the practical reading is that automated tracking could remove most of the manual matching workload in a screening programme, provided someone still reviews the high-burden scans. The failure mode is also reassuringly benign - false candidates that were never nodules - rather than missed real ones. This is retrospective, single-cohort and from one trial population, and the authors ask for prospective validation in diverse populations before it is relied upon.

• Route high nodule-burden scans for manual matching; performance fell to 72.8% above five nodules
• Expect pleural plaques to generate most automated matching failures - check them first
• Confirm which volume threshold the system uses; this study applied 100 mm3 solid component
• Automated matching is a workload tool, not a substitute for reading the scan
• Prospective validation in other populations has not been done

The statistics, in plain English

83.5% with an interval of 79.2-87.1% is the raw matching rate, and it is the wrong number to judge the system by - once the failures are classified, the clinically relevant figure is the 1.5% of persisting nodules requiring manual intervention, with an interval of 0.6-3.5%. The subgroup with more than five nodules made up just 6.6% of participants, so the 72.8% figure rests on few people and is imprecise. Everything here comes from one screening trial population, which was selected for high risk and is not representative of a general screening roll-out.

Read the rest in the app

You have read your two free briefings this month. The app carries all 27 specialties, every morning, free — and this finding is waiting in it.

Get Daily Doctor — free

Scan to keep reading on your phone. No account needed to start.