AI radiology summaries can miss small words with important meanings

A word such as “mild,” “severe” or “possible” can substantially change the meaning of a radiology report. Researchers led by Magdalini Paschali investigated whether large language models preserve such distinctions when producing explanations intended for patients. Their paper is an arXiv preprint submitted to Radiology: Artificial Intelligence for possible publication; the supplied evidence does not establish peer-reviewed acceptance.

The central question was not whether AI could diagnose a scan. Instead, it was whether a summarizer would accurately follow changes in the written report, without losing unchanged information or adding unsupported content.

To investigate, the team used adversarial testing: deliberately modifying inputs to expose weaknesses. Here, that meant controlled changes to medical language, not instructions designed to manipulate a chatbot. If “mild” becomes “severe” in the input, a faithful summary should convey the new severity.

The researchers assembled 13,801 de-identified report impressions from three public datasets: Merlin, INSPECT and MIMIC-IV. An impression is the radiologist’s concluding interpretation, rather than the entire report. The collection covered abdominal computed tomography, or CT; chest CT pulmonary angiography; and magnetic resonance imaging, or MRI, across several body regions.

A radiologist-validated dictionary contained 247 terms and alternative replacements. Substitutions targeted disease names, anatomy, location, severity descriptors, confidence language and examination types; measurements were altered numerically. Three substitution levels aimed for one, four or sixteen replacements where enough eligible terms existed. Some changes were clinically plausible, while others deliberately tested unusual or potentially erroneous wording.

A second strategy removed content. The system ranked impression points by their count of medical keywords, then removed the highest-ranking point or up to three points. For shorter impressions, it removed sentences or a medical keyword instead. Keyword density determined removal priority; it was not a direct measure of clinical importance.

Although edited variants were generated across the larger collection, the full summarization experiment used 300 reports: 100 randomly sampled from each dataset. Six models—GPT-5-mini, Claude Sonnet 4.5, MedGemma, Llama 3.1, Llama 3.3 and Qwen 3—received either detailed instructions or a concise request. The combinations produced 18,000 original-versus-edited evaluation pairs, comprising 36,000 summaries.

GPT-5 served as the primary automated evaluator, or “LLM judge.” It first extracted structured information about conditions, locations, severity and certainty. It then assessed preservation of unchanged facts, reflection of deliberate edits, preservation of diagnostic uncertainty and hallucinations. In this study, hallucinations included unsupported descriptions and recommendations, not just invented diseases. These dimensions were scored separately, so a summary could preserve existing facts yet still add unsupported material.

Under this judge, GPT-5-mini achieved the highest factual-faithfulness and certainty-preservation scores. With term substitutions, its factual-faithfulness scores were 85.1% with detailed instructions and 85.9% with concise instructions. Yet it fully reflected substitutions in only about 39–40% of cases, with another 42–43% rated partially reflected.

Hallucinations appeared in 28.1–30.1% of its substitution-test summaries, compared with 35.3–36.4% for Claude Sonnet 4.5 and 41.4–41.9% for MedGemma. Content removals were handled more robustly: GPT-5-mini’s hallucination rates were 14.0–18.2%. These are stress-test results, not estimates of error rates in routine patient care.

The category analysis showed why checking disease names alone is insufficient. For GPT-5-mini, 23.7% of anatomical substitutions, 22.3% of confidence-term substitutions and 21.2% of descriptor substitutions were missed, compared with 8.1% of pathology substitutions. Detailed prompting offered no consistent overall advantage; effects depended on the model and test condition.

The evaluation itself also had limits. Two resident radiologists independently reviewed 105 cases, all generated by GPT-5-mini with concise instructions. Overall judge–radiologist agreement was 0.55–0.56 on macro-F1, a measure balancing agreement across scoring categories, compared with 0.53 between the radiologists. Comparable agreement does not establish that the automated judge was consistently correct. Among 100 individually categorized hallucinations in a flagged subset, 45 were unsupported follow-up recommendations.

The synthetic edits cannot capture every real-world reporting problem. Only impressions were tested, each generation was run once, and 300 reports may miss rare findings. Readability, patient comprehension, follow-up behavior and clinical outcomes were not measured. The study therefore supports targeted accuracy checks—especially for anatomy, severity, uncertainty and added advice—but does not demonstrate readiness for unsupervised patient-facing use.