Better AI Does Not Automatically Make a Better Human–AI Team

AI accuracy is only part of the story when people work with an assistant. Advice can add useful information, repeat evidence someone already has, or receive too much or too little weight. In the preprint AI-Assisted Classification under Correlation Neglect and Trust, Dingwen Kong, Liang Lyu, Saurabh Amin, Amine Bennouna, Daniel Huttenlocher and Asuman Ozdaglar build a mathematical framework around those distinctions and examine it using data from a crowdsourced question-answering study.

Their central argument is conditional, not a claim that stronger AI generally harms people. Under particular combinations of shared evidence and human behavior, improving an assistant’s standalone capability can reduce the team’s accuracy. The empirical analysis, meanwhile, finds large benefits from assistance in the setting studied.

When two sources repeat the same evidence

The model describes a person making a binary decision using personal evidence and an AI signal. It separates three ingredients: AI capability, the proportion of the AI’s evidence already available to the person, and the person’s behavior. Behavior has two components: awareness of that overlap and the weight placed on AI information, which the authors call trust.

Correlation neglect means treating overlapping evidence as though it came from independent sources. A person may effectively count the same clue twice. Within the model, sufficiently high overlap can make a more capable assistant counterproductive for a worker who neglects that dependence.

The authors distinguish three regions: complementarity, where the team performs best; impairment, where the unaided person performs best; and automation, where AI alone performs best. These are accuracy comparisons within the framework, not complete assessments of whether a workplace should automate.

The theory also complicates the idea that increasing trust is always desirable. Discounting advice can partly protect someone against double-counting shared evidence. The accuracy-maximizing level of trust therefore depends on the deployment rather than following one universal target.

What the workers actually did

The empirical analysis uses ChatBench, a study involving multiple-choice questions in mathematics, physics and moral reasoning. Workers answered first without assistance, stated their confidence, then saw an AI response and submitted a final answer. Each worker was randomly assigned either GPT-4o or Llama-3.1-8b.

Although the original study included 643 workers, the paper’s main analysis uses a filtered sample of 2,199 paired observations from 315 workers across 389 questions. These observations had matched unaided answers and a definite AI letter recommendation.

The authors report that pooled accuracy rose from 61.2% without assistance to 82.4% with it—about 21 percentage points. But improvement over people alone did not necessarily mean improvement over AI alone.

For GPT-4o, assisted workers achieved 85.1% accuracy, compared with 88.4% for the AI’s recommendations alone. That difference was statistically significant. For Llama-3.1-8b, the team scored 79.5% and AI alone 78.7%; the difference was not statistically significant. The authors describe the latter as a boundary between complementarity and automation, rather than evidence of a clear team advantage.

Their fitted behavioral model estimates a trust parameter of 0.285, interpreted as applying roughly a quarter of the model’s Bayes-rational weight to AI information. That is a model-based estimate, not a direct reading of workers’ attitudes.

Simulated reliance is not an intervention

The authors also simulated a rule for choosing between the person’s original answer and the AI’s recommendation when they disagreed. It used human confidence and question-level AI accuracy, with correctness probabilities estimated through cross-validated logistic regression.

For GPT-4o, the simulated rule increased accuracy from 85.1% to 87.7%, a gain of 2.6 percentage points. For Llama-3.1-8b, it reduced accuracy by 1.4 points. These are retrospective policy simulations, not measured improvements following a trust-building intervention with workers.

The distinction matters because people saw the AI’s actual response, while the simulated rule used only two coarse signals. The authors suggest that response-specific information may help explain why workers outperformed the rule with the weaker assistant.

Where the evidence stops

The theoretical results depend on a stylized binary model with Gaussian evidence and a simplified rule for combining signals. The empirical task involves multiple-choice questions, and its estimated signals are proxies. Human confidence has only three categories; AI question-level accuracy averages across repeated generations rather than capturing each response’s reasoning.

Parameter uncertainty also matters. Two estimation approaches produce similar trust estimates but substantially different estimates of correlation awareness, and reverse the ordering of overlap between the two AIs. The authors favor the direct covariance-based overlap measure but caution against treating its values as precise structural estimates.

The supplied PDF extraction also disrupts equations and table layouts and does not preserve the figures as viewable graphics. The account here therefore relies on explicit prose and readable numerical comparisons, not reconstructed visual boundaries.

The practical implication is to evaluate the whole decision process, not just the assistant’s benchmark score. This preprint offers a framework for doing so; it does not establish that its trust estimates or automation recommendations transfer to medicine, hiring or other high-stakes settings without domain-specific validation.