Can AI Agents Draw Reliable Health Conclusions? A Benchmark Says Not Yet

Scientific AI agents are increasingly asked to find research, connect evidence across sources and summarize conclusions. In health, that kind of synthesis could inform consequential decisions, making a polished-sounding answer insufficient: it also needs to be factually correct and cover the relevant findings. Hayoung Jung and colleagues introduce SciConBench to test how well agents perform this task.

The benchmark contains 9.21K questions paired with expert-written conclusions from systematic reviews, according to the paper’s abstract. The authors describe it as a large-scale, live benchmark spanning health and medical domains. Instead of treating each conclusion as one indivisible answer, their evaluation pipeline breaks it into atomic facts—individual claims that can be checked. The system then measures factual precision and recall: broadly, whether the facts an agent states are correct, and how much of the expected information it includes. Those measures are combined into factual F1, a single score that reflects both correctness and coverage.

A central concern is data leakage. If an agent has encountered the answer or closely related material during training, an evaluation may mistake familiarity for the ability to retrieve and synthesize evidence on demand. To address this, the researchers introduce SciConHarness, a “clean-room” setup that gives agents controlled web interaction. They evaluate eight frontier models and deep-research agents, and compare results under clean-room and unconstrained conditions.

The abstract reports that factual quality remains low in the clean-room tests: the best agent achieves a factual F1 of 0.337. The authors also say clean-room evaluation consistently lowers performance compared with unconstrained evaluation, which they interpret as evidence that leakage can inflate estimates of agents’ synthesis ability. This is a warning about measurement as much as about models: a score can reflect the evaluation environment, not just the underlying capability being tested.

The authors also audit consumer-facing tools, including Google AI Overview and OpenEvidence. They report that these systems frequently produce incomplete conclusions and sometimes contradictory ones, even when the ground-truth answer is available. That observation points to two distinct failure modes. An answer may include claims that do not fit together, or it may omit parts of the evidence needed for a faithful synthesis. The abstract does not provide detailed results for each product, so it is not possible from the available evidence to compare them or quantify how often either failure occurs.

These results should be read with care. The available source here is the paper’s abstract, not its full text. It does not specify how the questions are distributed across medical topics, how expert conclusions were selected, how the automated fact-checking pipeline was validated, or how performance varied among the eight models and agents. Nor does the abstract establish whether the benchmark captures every kind of evidence-synthesis task that clinicians, researchers or patients may encounter. Those details matter when deciding how far to generalize a benchmark score.

The reported F1 score is not a measure of clinical outcomes, and the study does not show that using these agents causes harm or improves care. It evaluates factual synthesis against expert-written conclusions. Likewise, the comparison between clean-room and unconstrained settings supports concern that leakage affects measured performance, but the abstract alone does not tell us how much each specific source of leakage contributes.

The study’s broad message is therefore narrower—and more useful—than a verdict that AI can or cannot answer health questions. On this benchmark, agents struggled to reproduce scientific conclusions completely and accurately, and evaluation conditions changed the apparent level of performance. The authors argue that clean-room testing is essential for assessing open-domain agents. Until the full methods and results can be examined, the benchmark offers a cautionary signal rather than a definitive ranking or a clinical readiness test. The paper is an arXiv preprint; the supplied source does not establish whether it has undergone peer review.