An AI coordinator improves answers by directing attempts, not solving problems

Spending more computation wisely

A language model can spend additional computation answering a question without being retrained. Researchers call this test-time scaling. One approach repeatedly critiques and revises an answer. Another generates several independent answers, then selects or combines them. Both face a practical problem: producing a correct answer somewhere along the way does not guarantee that it becomes the final response.

Macardle Meng and Haomin Zhang propose ODSA, short for Orchestrated Discovery and Self-Aggregation, to manage that process dynamically. Their paper is an arXiv preprint, submitted on November 8, 2026; the findings should be read as reported experimental results rather than settled conclusions.

ODSA gives an orchestrator model responsibility for deciding which attempts to commission, which existing solutions to compare and when to stop. It does not divide a question into separate subtasks. Workers tackle the original problem, while the orchestrator allocates their effort.

Three actions, one continuing conversation

The orchestrator keeps a running conversation containing the investigation’s history. Workers are fresh model calls without that continuing memory; they receive the problem and the material needed for their assigned action.

There are three actions:

• Discover: request an independent solution along a specified approach, ideally different from those already tried.
• Compare: give a worker selected candidate solutions and ask it to work through their disagreements. It can produce a new answer rather than merely choose an existing one.
• Accept: propose a candidate as final, triggering a verifier instructed to hunt for mistakes, counterexamples or boundary cases.

If the verifier disputes a proposal, its corrected answer joins the candidate pool and the investigation continues. A verifier’s objection is therefore another solution to examine, not an automatic verdict.

Every role uses the same underlying model in a given experiment. The reported gains do not come from adding a more capable model as the critic.

For short-answer questions, an extra summarizer condenses workers’ reasoning into numbered steps. This gives the orchestrator and comparison workers more information than a bare answer without filling the context with complete derivations. For coding questions, the candidate is the program itself. Crucially, programs are not executed during the investigation: only the final program is run for benchmark scoring.

Gains on small benchmark samples

The authors tested Gemma 4 31B-IT, GPT-OSS-120B and DeepSeek V4 Flash, using three fixed random seeds.

For Humanity’s Last Exam, they sampled 50 questions from the text-only portion of an audited Gold subset. A fixed model judge compared final answers with reference answers. They also sampled 50 programming problems from LiveCodeBench v6, where correctness meant passing the benchmark’s tests.

On the exam questions, ODSA improved accuracy over a single direct model call by 4.7 to 18.0 percentage points. Gemma showed the largest increase, from 42.7% to 60.7%; DeepSeek rose from 62.0% to 75.3%.

On coding problems, improvements ranged from 0.6 to 14.0 percentage points. Gemma increased from 81.3% to 95.3%. DeepSeek, already at 98.7%, reached 99.3%, leaving little room for improvement.

Those single-call comparisons do not isolate the value of orchestration from the value of spending more computation. The authors therefore also compared ODSA with approximately cost-matched methods, including repeated self-refinement, answer selection and answer aggregation. ODSA was best or tied for best in five of six model–benchmark combinations. On GPT-OSS exam questions, however, both selection and aggregation performed better.

Finding an answer is not enough

The authors measured “conversion”: how often a method returned a correct final answer when its candidate pool contained one.

On the exam sample, ODSA converted such opportunities into correct answers 80.6% to 81.9% of the time, the highest reported rate for each model among the methods compared. Comparison workers produced roughly 49% to 77% of ODSA’s final answers. This analysis points toward handling existing candidates as an important part of its performance, although it does not establish that every component is necessary.

The limits of coordination

ODSA cost 9 to 22 times as much as a single call. Its spending also varies with the investigation, making costs harder to predict than a fixed number of attempts.

Termination rules matter: experiments used a 20-call worker budget and five permitted acceptances. A dispute refreshes the worker budget, while the last permitted acceptance ends without verification. Verification is therefore neither universal nor a guarantee of correctness.

Component-removal experiments also gave mixed results. Removing verification hurt two models but helped Gemma; removing discovery directives helped GPT-OSS but hurt DeepSeek. Many differences were smaller than the variation across seeds, and these experiments were not cost-matched.

The evidence supports a bounded conclusion: on these small samples, coordinated attempts improved on single calls and competed well with other ways of spending additional computation. It does not establish a universally superior strategy.