Why more AI tools alone may not deliver complete radiology reports

A report is more than a detection task

Radiologists do not simply look for one disease. They survey multiple organs, measure abnormalities, compare earlier examinations and explain what the findings mean together. A specialist AI tool might handle one step well without covering the rest.

Aubin Charley and colleagues investigate whether an “agentic” system can bridge that gap. Here, a language model acts as a coordinator: it plans an examination’s analysis, requests specialist tools and combines their responses into a structured report. The study separates two questions: are the necessary tools available, and does the coordinator use them effectively?

Testing the coordinator, not the scanners

The researchers describe a shared interface for tools that detect abnormalities, outline structures, calculate measurements and support other reporting tasks. Their pipeline plans work, groups independent tasks for parallel execution, combines results, drafts a report and reviews it.

Crucially, the experiment did not process medical images or run commercial devices. Tool responses were simulated from existing reference reports. This deliberately removed real detectors’ mistakes, allowing the researchers to examine tool coverage and coordination under idealized conditions.

The evaluation used PARROT, a collection of fictional radiology reports. The main comparison included 1,059 English-language reports with a clinical indication—the reason for ordering the examination—and usable outputs from all four experimental conditions. These covered several imaging methods and body regions.

One tool catalog contained 1,989 classes of findings or measurements, consolidated using language models from documentation for 1,164 FDA-listed radiology AI devices, with decisions through March 2026. Each class was available only for its listed imaging methods. An alternative, unlimited catalog allowed requests for any plausible tool and finding class.

The coordinator either planned from the clinical indication and a reporting template, or received the target report itself. The latter deliberately supplied the answer: it was an experimental upper-bound comparison, not a realistic clinical workflow.

What the system recovered

The headline measure was “actionable recall”: the proportion of reference findings classified as critical, urgent or notable that appeared in the generated report. This is a measure of finding coverage, not overall diagnostic accuracy or patient benefit.

With the restricted catalog and indication-based planning, recall was 29.0%. Allowing unlimited tools raised it to 43.7%. Giving the planner the target report increased those figures to 42.6% and 74.0%, respectively.

Thus, both tool availability and information used in planning mattered. Even with unlimited tools, idealized responses and advance knowledge of the report, the pipeline failed to recover roughly one-quarter of actionable findings.

The researchers also classified 9,675 missed or incorrectly reported findings. This analysis was not limited to the headline actionable subset. A language-model judge attributed 61.8% to missing catalog classes for the relevant imaging method. Another 31.9% involved available classes or tools that were not requested. Most remaining misses required a different kind of tool.

Adding tools helped—but coordination still mattered

The team identified recurring missing tools in 659 reports, then tested expanded catalogs on 100 separate held-out reports. On this smaller sample, baseline recall was 34.7%, compared with 58.0% for unlimited tools. These figures belong to the held-out experiment, not the full-corpus comparison.

Adding 99 recurring missing tools raised recall to 51.9%, closing an estimated 74% of the gap to the unlimited catalog. That does not mean recovering 74% of all findings. The uncertainty interval for the gap closed was wide, from 51% to 105%.

Expanding further to 402 added tools produced 53.4% recall, without a clear additional gain. Actionable precision—the share of generated actionable findings matched to the reference—was 52.6% with 99 additions, falling to 47.5% with 402.

The coordinator also mattered. Two smaller models given unlimited tools achieved roughly 35.5–35.6% recall, close to the main model’s restricted-catalog baseline. One produced no report for ten examinations, which were scored as empty.

What remains unproven

These are simulations, not evidence of clinical readiness. Fictional reports cannot establish performance in real patients, and simulated tools exclude missed detections and false alarms. Broad tool classes may also return more information than real devices would.

Language models constructed the catalog, scored reports and attributed misses, introducing additional uncertainty. Reference findings were re-extracted during scoring, so denominators varied slightly between conditions. The framework also excluded an image-reading model as a fallback.

The work is arXiv preprint 2611.09836v1, submitted on 6 November 2026; the supplied landing record lists no journal publication or peer-review outcome. Its supported conclusion is narrower than autonomous reporting: broader tool coverage and better coordination are complementary requirements, and neither alone made this simulated pipeline comprehensive.