Design: Retrospective feasibility study with expert rating
Population: 360 Dutch primary care X-ray and ultrasound reports
Primary outcome: Accuracy, completeness, readability and hallucinations
Effect: Conclusions met quality criteria in 96.0%; hallucinations in 12.5%

A Dutch study used a locally hosted large language model to rewrite 360 primary care X-ray and ultrasound reports into plain language, avoiding sending text to the cloud.

Rewritten conclusions met predefined accuracy and completeness criteria in 96.0%, and full reports in 71.0%. Readability improved markedly, though reports grew by a median of 65 words. Clinically relevant hallucinations — statements not supported by the original — occurred in 12.5% of reports.

Patients increasingly read their own reports, and plain-language versions are an obvious use for these tools. But a one-in-eight error rate means they cannot go to patients unchecked.

Do not release AI-generated patient summaries without clinician review
Start with conclusion sections, which were rewritten more accurately
Check rewritten text against the original for added findings or reassurance
Keep patient data on local systems when testing these tools

Why it matters

Patient-facing AI summaries are arriving, and their error rate is too high for unsupervised use.

The statistics, in plain English

12.5% hallucinations means 45 of 360 reports contained a relevant statement not in the original. Quality was rated by experts on a 5-point scale, not by patients.