Choosing the right tool before analyzing an image: ZETA tests scientific AI routing

A tool menu is not a selection rule

An AI agent can have access to the right scientific tools and still choose the wrong one. A request to locate a crack calls for detection; a request to outline it calls for segmentation. Both actions may be permitted, but only one matches the requested operation.

That distinction is the focus of Route First, See Later, a preprint by Shubhabrata Mukherjee, Jeffrey Donatelli and Daniela Ushizima, submitted to arXiv on November 4, 2026. The paper introduces ZETA, an architecture that separates interpreting a scientific request from executing the selected visual task.

The authors distinguish an action space, the menu of available operations, from a routing policy, the rules connecting requests to those operations. Restricting an agent to valid actions prevents choices outside the menu. It does not necessarily prevent a valid but inappropriate choice.

Separating the decision from the image analysis

ZETA follows a “Route First, See Later” design. Its router selects an operation using the natural-language request and available non-pixel context, such as metadata or domain constraints. The corresponding capability is then invoked. The current action space includes text-based explanation, image description, detection, counting and segmentation.

This separation makes two kinds of failure distinguishable. A routing failure occurs when the agent selects the wrong operation. An execution failure occurs when it selects the correct operation but the downstream tool fails to perform it successfully.

For example, a segmenter might produce a plausible boundary even though the user requested only localization. Conversely, the router might correctly choose segmentation while the segmenter misses the target. Looking only at the final answer can obscure that difference. ZETA makes the selected route, invoked tool and resulting artifacts separately traceable.

The architecture also treats measurement differently from free-form answer generation. Quantities are computed through deterministic operations on compatible visual artifacts, such as segmented regions or validated geometry. This provides a reproducible calculation from an explicit representation, but it cannot repair an incorrect boundary supplied upstream.

What the experiment changed

The researchers compared two zero-shot routing conditions across five models using 51 distinct scientific-intent queries. The paper explicitly notes that this is a query count, not a count of unique images.

In the first condition, models received the legal routes, measurement options and output contract, without explicit semantic rules mapping requests to routes. In the second, the researchers added those intent-to-route rules.

The available actions and measurement options remained unchanged. For each comparison, the model, queries, route interface, output contract and inference configuration were otherwise held fixed. The experiment therefore examined routing-policy specification rather than a change in visual tools.

Large gains, but not for every model

For Qwen 3.8, reported segmentation routing accuracy increased from 6.2% to 75.0% with explicit guidance. Detection routing accuracy increased from 60.0% to 100%. These percentages describe choosing the correct operation—not detecting or segmenting image content correctly.

Nemotron 3.5 also improved: segmentation routing rose from 37.5% to 87.5%, while detection routing rose from 70.0% to 90.0%. Qwen 3.6 showed gains in both categories as well.

However, clearer rules were not uniformly beneficial. Qwen3 1.7B declined across several categories, including description routing, which fell from 100% to 0%. For muse-glimmer 30b, segmentation routing improved from 0% to 43.8%, while detection routing declined from 90.0% to 80.0%. Counting routing remained at 100% for all five models in both conditions on this benchmark.

These mixed results matter: providing a policy and having a model interpret it successfully are separate challenges. The study does not establish why individual models responded differently.

What the findings do—and do not—show

The preprint presents a preliminary, small evaluation of routing correctness. It does not establish downstream visual accuracy, measurement accuracy or end-to-end scientific task success. Its model-dependent results also limit general claims about the benefits of explicit instructions.

The supported conclusion is narrower: the rules for selecting tools can materially affect routing even when the tool menu stays fixed. ZETA makes that selection stage separately inspectable. This supports more precise failure attribution, but traceability alone does not establish that an image-analysis result or scientific measurement is correct.