Choosing when one AI model is enough—and when to ask several

Language-model services face a recurring choice: use a cheaper model, pay for a more capable one, or ask several and select an answer. The best option may change from question to question. An expensive model need not be best at every task, and a larger ensemble need not justify its additional cost.

In a November 2026 arXiv preprint, Saul Urso and colleagues introduce two systems for making that choice. PISA selects a single model for each prompt. Its extension, PISA-E, selects both which models to ask and how many. The study reports benchmark experiments, not a demonstrated production deployment, and the supplied source is a preprint rather than an identified peer-reviewed journal publication.

Predicting capability before asking

Both systems learn from historical prompts, model responses and evaluation scores. First, an embedding model converts each prompt into a numerical representation of its meaning. A clustering method identifies regions containing similar prompts.

Rather than assigning a new question to just one cluster, PISA gives it degrees of membership across several clusters. A question can therefore draw information from multiple neighboring regions. Separate statistical predictors then estimate each candidate model’s likelihood of success. Training labels classify benchmark scores of at least 0.5 as successful, even though some benchmarks originally provide graded rather than binary scores.

PISA balances those estimates against each model’s average training-set invocation cost. A user-adjustable parameter controls the emphasis on predicted quality versus cost. Changing that preference does not require retraining the capability predictors.

Selecting a team—and one final response

PISA-E reuses the same predictions to evaluate possible teams. Its selection score estimates the probability that a strict majority of the chosen models will answer correctly, assuming their correctness events are independent once the prompt is given.

That assumption makes the calculation manageable, but the score is a proxy—not a guarantee about the final answer. Models can share mistakes, and choosing the right response remains a separate problem.

The system also avoids searching every possible combination. It ranks models by their individual cost-aware scores, then evaluates progressively larger teams from the top of that ranking: one model, three, five, and so on, up to the configured maximum. Experiments allowed up to eleven. This shortcut reduces search but does not guarantee the best possible subset under its scoring rule.

Selected models are queried in parallel. A response selector compares the meaning of their answers using BERTScore, weighting both candidate answers and their agreement with others by predicted model capability. It returns one existing response; it does not write a synthesis or run a debate.

What the benchmarks showed

The researchers evaluated four benchmarks: LLMRouterBench Track-2, RouterBench, FusionBench and SPROUT. They tested both familiar task distributions and held-out task groups absent from router training. Experiments used benchmark-provided responses and five random seeds with fixed data partitions.

For single-model routing, PISA achieved the highest average hypervolume among the tested methods in both settings. Hypervolume summarizes the quality–cost trade-off across operating points, rather than measuring answer accuracy at one setting. PISA led in six of eight benchmark–setting combinations, not all eight.

With quality given full priority, PISA-E’s reported average task scores were 75.27% for familiar distributions and 63.24% for held-out groups. These exceeded the strongest competing ensemble scores by 2.09 and 2.87 percentage points, respectively. The figures combine benchmark-specific evaluation scores; they are not a universal percentage of questions answered correctly.

At selected lower-cost settings, PISA-E also surpassed the strongest competitors’ average task scores while spending less on model calls. In the familiar-distribution setting, it scored 73.78% versus SelectLLM’s 73.18%, at approximately 36% lower reported cost. For held-out groups, it scored 60.85% versus DFPE’s 60.37%, at approximately 25% lower cost. Those savings belong to these particular comparisons, not every operating point.

Where the evidence stops

The cost accounting excludes local routing and response-selection overhead, so the results do not establish equivalent savings in total operating expense. The highest-quality setting was also more expensive than the strongest competing ensemble in each setting.

Adaptive team size improved the aggregate quality–cost trade-off over fixed-size variants, but single-model routing remained better on some benchmarks. A dependency-aware alternative did not consistently improve results with the default response selector, though that does not establish that model errors are independent.

The practical finding is narrower than “more models are better”: these experiments support allocating extra model calls selectively, using learned estimates of where they may help.