Can faster AI models capture the economic choices people disagree on?

Modelling disagreement, not just the average

In a November 2026 preprint, Maxime Saxena, Fabio Caccioli and R. Maria del Rio-Chanona compare 10 decision models with eight large language models across household spending questions, economic games and job choices. Their central finding concerns diversity: the leading decision models generally reproduce the distribution of human answers more closely than the language models tested.

That matters for economics because a population is not simply its most common response. Households can react differently to income changes, while workers attach different values to flexibility. Simulations that compress those differences into one dominant answer risk missing an important part of economic behaviour.

Decision models take a scenario, a question and a fixed menu of answers, then assign a probability to each option. Unlike a language model producing a written response, they return the whole choice distribution directly. Researchers can use those probabilities to generate varied decisions in a simulated population.

The authors designate four models as “frontier” systems using general decision-making benchmark scores: Jev, D1 Ultra, Mammoth and Perplexity. That label describes their benchmark standing, not a prior guarantee of human realism.

Three tests of economic behaviour

The spending exercise uses 1,000 households randomly selected from the New York Federal Reserve’s 2020–2024 survey data. Models receive demographic personas and questions about how households would respond to an unexpected 10% increase or decrease in future income, choosing among seven spending, saving and debt responses.

All four frontier decision models fit the aggregate human answer distribution better than every language-model configuration tested, according to the paper’s distributional measure. Their ability to capture the most common human answer was broadly comparable, with mixed statistical differences.

The second exercise covers five economic games and seven player positions, testing behaviour involving generosity, fairness, trust, cooperation and risk. Here, language models were better at concentrating on the most typical human responses. But frontier decision models were closer to the overall human distribution in 79 of 80 comparisons using the authors’ main distance measure; the remaining comparison was not statistically significant.

These findings are compatible. A system that repeatedly picks the most popular answer can score highly on the paper’s “Turing Test” while overlooking everyone who chooses differently. The test measures how typical an answer is under the human distribution, rather than whether a conversation fools an observer.

There is another qualification: some frontier decision-model configurations performed worse than a simple uniform distribution on the games’ distributional measure. Beating language models therefore did not consistently mean beating an equal-probability baseline.

The labour exercise recreates choices from a call-centre recruitment field experiment. Applicants chose between standard office work and alternatives involving flexibility, home working or employer-controlled schedules, with different wage gaps. Frontier decision models better reproduced the gradual change in human choices as relative pay changed. Language models tended to switch more abruptly between jobs.

Cheaper choices, conditional projections

The researchers also report substantial speed and price advantages. Across their experiments, Jev averaged about 0.24 seconds per call, compared with 1.71 seconds for Sol and 6.10 seconds for Gemini Argon. Perplexity’s estimated API charge averaged about $0.006 per 1,000 calls, versus $0.871 for Sol.

Those differences could make larger simulations practical, but the paper’s large-scale examples are projections, not demonstrated economic simulations. They assume similar prompt lengths and performance, ten concurrent requests, and specified numbers of decisions. Some applications require multiple decision-model calls for each language-model call. Reported zero prices for locally run models also do not represent comprehensive computing costs.

What remains unproven

The preprint does not establish that these systems can replace human participants or reliably predict policy effects. Spending answers describe intentions, not observed purchases. The spending and labour datasets are US-based, limiting conclusions about other populations. The authors also identify likely training-data contamination because the underlying studies were published before the models were trained.

Measurement choices matter too. Game answers were grouped into ten bins, and decision-model probability distributions were compared with distributions formed from repeated language-model answers. Labour comparisons used fitted curves corrected for estimated inattention; most AI systems did not reproduce the random choices attributed to a substantial share of human applicants.

The authors call for validation using unpublished experiments and culturally diverse populations. Their evidence supports a narrower conclusion: for these tasks and evaluation methods, leading decision models offer a faster, often cheaper way to represent variation—not proof that an artificial population behaves like a real economy.