AI safeguards caught some radiology paperwork errors—but plausible misinformation still slipped through

When answering becomes acting

A chatbot’s incorrect answer can be discarded. An AI agent—here, a language model that uses software tools—can instead save that answer into a record or mark unfinished work as complete. This study examined that distinction in radiology documentation, not image interpretation.

The researchers asked how often six agents executed clinically important false or unsupported statements, and whether safeguards placed immediately before record-changing actions reduced those errors. The work is an arXiv preprint, posted November 8, 2026; it should be treated as preliminary research rather than established clinical evidence.

A synthetic radiology department

The team created 48 fictional scenarios, reviewed and revised by three radiologists. They covered three jobs: documenting findings and their communication, preparing the radiology section of a multidisciplinary cancer-meeting draft, and updating follow-up recommendations.

Every scenario had a clean version. Forty-three also had a version containing a plausible misleading statement in an outside or referring clinician’s document, making 91 variants. Examples included an external letter naming the wrong side of a finding or reporting that an examination had happened without supplying its report.

These were factual distractions, not hidden commands telling the agent to ignore its instructions. The experiment therefore tested misleading information, not deliberate prompt injection.

Six models completed every variant under three configurations, producing 1,638 main evaluation episodes. The models were Claude Sonnet 5.5, GPT-6.1-Sol, GLM-5.3, MiniMax M3, Gemma-4-31B and Qwen3.8-27B. All received the same task instructions and tool budget.

Two layers before a write

The baseline included permissions and ordinary system checks. A second configuration added 11 deterministic rules: fixed checks for issues such as outdated sources, mismatched examinations and claims of communication without a documented event.

The third added a locally run Gemma-4-31B checker. It examined whether proposed actions were supported by the available records, including their certainty, timing and handling of contradictions. Blocked actions returned feedback so agents could revise them.

A separate, non-LLM evaluator compared executed structured statements with a radiologist-defined reference inaccessible to both agents and checker. The main outcome was an episode containing at least one high-severity violation: an error judged capable of immediately affecting care in that fictional scenario. It was not a measurement of actual patient harm.

Fewer serious errors, not their elimination

In the exploratory analysis averaged across the six agents, the estimated proportion of episodes with high-severity errors was:

• 19.4% with permissions only;
• 16.7% with deterministic rules;
• 13.2% with rules plus the AI checker.

Together, the safeguards reduced the estimate by 6.2 percentage points, roughly one-third relative to baseline. The 95% confidence interval indicated a reduction between 3.0 and 9.5 percentage points. Adding the checker to rules alone reduced it by 3.5 points.

Rules alone had a less certain overall effect: their confidence interval included no change. Individual-agent estimates were also too imprecise to establish the overall incremental reductions separately for each model. The study did not detect differences in safeguard effects between agents; that does not establish equal effectiveness.

Task success did not decrease detectably. Nevertheless, with both safeguard layers, the pooled high-severity error estimate for misleading-information variants remained 17.4%.

Reading the source was not enough

Across misleading-information episodes, agents opened the altered document 83% of the time, equally often in episodes with and without a high-severity error. Among written statements with high-severity errors in those variants, 156 of 268—58%—cited that document.

This suggests that many failures involved misjudging information already encountered, rather than simply failing to retrieve it. It does not prove that every error was caused by the misleading document.

The safeguards also imposed friction. Between 64% and 68% of blocked actions would have complied with the reference standard. Revision opportunities helped agents recover, but such unnecessary blocks could require extra work or human review in practice. Even a justified block did not guarantee an error-free episode.

What this cannot establish

All cases were synthetic, German-language and text-only, in one simulated environment. Their error rates cannot be read as hospital error rates, and reduced documentation errors do not demonstrate improved patient outcomes.

The evaluator primarily scored structured statements; its episode-level judgments were not independently checked by radiologists. The strongest conclusions rely on exploratory pooling across only 48 base scenarios. API-based agents were run once per configuration, limiting assessment of variability.

The practical implication is narrower than deployment readiness: checking writes can help, but a valid citation and a permitted action do not guarantee clinically correct content. The authors propose stronger handling of conflicting sources and targeted human confirmation before closing tasks—approaches this experiment did not itself validate.