- Design
- Prospective intra-individual reader study with eye tracking
- Population
- 10 radiology residents reading 33 chest radiographs per session
- Primary outcome
- Diagnostic accuracy (kappa vs expert reference) before and after night shift
- Effect
- Fatigued readers kappa 0.71 to 0.61 (p < 0.001); AI Δkappa 0.03–0.04 (p ≥ 0.525)
In this prospective reader study, ten radiology residents read sets of 33 chest radiographs before and after a first-call on-site night shift, with and without an AI assistant. Eye tracking measured oculomotor efficiency to define fatigue objectively.
Six of the ten met the fatigue definition. Their agreement with an expert reference fell from kappa 0.71 to 0.61 after the shift; non-fatigued readers fell less (0.72 to 0.65). Reporting time per radiograph did not change. The AI assistant used did not significantly improve accuracy (change in kappa 0.03–0.04), though it shortened reading times in non-fatigued readers.
The important finding is the combination: accuracy dropped while speed held, so fatigued readers would not notice their own decline from their throughput. The study is small and from one centre, but it supports structured safeguards for post-night reporting rather than reliance on AI to compensate.
- Avoid scheduling complex or high-volume reporting immediately after a night shift.
- Do not assume steady reporting speed means steady accuracy.
- Consider second reads for urgent overnight chest films where workload allows.
- The AI tool tested did not restore post-shift accuracy; do not rely on it as a fatigue safeguard.
Why it matters
Fatigue erodes accuracy silently, because readers keep the same pace.
Don't overread it
Ten residents at one centre; results may differ for consultants or other AI tools.
The statistics, in plain English
Kappa measures agreement beyond chance; a fall from 0.71 to 0.61 moves from substantial towards moderate agreement. With ten readers, the estimates are imprecise, but the fall was seen in both fatigued and non-fatigued groups.
Read the rest in the app
You have read your two free briefings this month. The app carries all 27 specialties, every morning, free — and this finding is waiting in it.
Scan to keep reading on your phone. No account needed to start.
Tomorrow morning, before your first patient
One edition a day for radiology, written by the desk, every claim tied to its paper. Six minutes.
Get the app — free