Scripted automation beats an AI agent on a stable laboratory portal

A preprint posted to arXiv on November 2, 2026 compares two ways of retrieving mammography and breast ultrasound reports from a laboratory website. Its central finding is narrow but practical: on the single, stable portal tested, explicitly programmed automation outperformed a language-model-based agent in speed, operational cost and report-selection accuracy.

The work concerned collecting documents, not interpreting medical findings. Neither system performed automated clinical inference.

The same task, two approaches

The case study came from ClickPalm, a personal health record platform. The reports themselves were retrieved from a separate laboratory portal because the two systems were not integrated. Previously, the responsible physician downloaded reports manually.

The researchers built a traditional robotic process automation system, or RPA, using Python and Playwright. Its browser interactions were explicitly programmed. The competing agent process automation system, or APA, used browser-use, LangGraph and Gemini 2.5 Flash-lite to decide how to navigate the website from natural-language instructions.

Importantly, the agent was not responsible for every operation. Critical filtering, document capture and duplicate-control functions relied on deterministic tools. Both systems applied the same business criteria and maintained a browser session across each patient batch. This was therefore a comparison of two engineered workflows, not a standalone language model against a script.

The task was to locate patients, select eligible reports from 2024–2025 and save them as PDFs, while excluding unrelated documents such as pre-operative localization letters. Those letters could have names resembling the target reports, making document identification a substantive part of the task.

What the experiment measured

The study processed 69 patients, corresponding to 71 eligible exams, in two rounds seven days apart. Four patients had no active laboratory registration. Daily API quotas for the agent required processing in batches over five consecutive days; both systems handled the same batches on the same dates. Both ran in containers on AWS.

Across the two rounds, each system made 456 exam-level decisions, including whether to collect eligible reports and reject other documents or duplicates. Evaluation also incorporated automatic duplicate checking: downloaded documents with at least 90% textual similarity were treated as the same report.

The scripted system correctly collected 142 eligible reports across the rounds, missed none and made five false-positive collection decisions. The agent correctly collected 137, missed five and made 11 false-positive decisions.

Reported accuracy was 98.90% for RPA and 96.49% for APA. Accuracy includes both correct downloads and correct decisions not to download. F1-score, which balances precision and recall, was 98.27% and 94.48%, respectively. These are document-collection metrics, not measures of diagnostic accuracy.

A large speed and cost gap

Average processing time was 16.07 seconds per patient for the scripted workflow, compared with 207.21 seconds for the agent—approximately a thirteenfold difference.

Reported operational cost per successfully downloaded report was $0.030 for RPA and $0.187 for APA, making the agent about 6.2 times as expensive. The calculation included infrastructure and, for the agent, language-model API charges. It was not a full accounting of development and maintenance costs. The authors identify separating API expenditure from infrastructure expenditure as future work.

The paper also reports 100% login, patient-search and complete-download rates for both systems, with search and download rates restricted to the 65 registered patients. Those workflow figures should not be mistaken for error-free exam collection: the separately reported classification results include five missed eligible reports for the agent.

Repeated execution also exposed differences. The agent’s recall rose from 92.96% in the first round to 100% in the second. Its F1-score varied by 5.59 percentage points, compared with 0.68 points for RPA. The causes of the agent’s variation remain unresolved.

What the result does not establish

The laboratory portal underwent no structural layout changes during collection. Consequently, the experiment could not directly establish whether an agent’s adaptability would compensate for its higher running cost when interfaces change.

The comparison also covered one portal, two collection rounds and one principal language model. The authors call for tests involving more variable interfaces and different models.

For this stable workflow, the evidence favors scripted automation. It does not establish that scripts will always outperform agents, nor that agents will deliver maintenance savings elsewhere. Those questions require experiments that actually expose both approaches to changing interfaces.