When AI agents mistake a signal for consensus

Testing a supposed group view

Researchers studying multi-agent language models constructed a benchmark in which four model instances make separate decisions on the same scenario. The agents then see ambiguous comments attributed to peers and make public decisions. The preprint calls a mismatch between public convergence and recorded private judgments “phantom consensus”. This is an experimental analogy to a social phenomenon, not evidence that language models possess human beliefs.

The benchmark uses 480 scenarios across five decision environments. Each asks agents to choose among proceeding, blocking, or deferring for review. The researchers compare each agent’s independent choice with its public choice and its reported estimate of what the group privately chose. These reports let them compare the apparent group view with the private decisions actually recorded.

The team also tested alternative conditions: neutral public comments, explicit peer recommendations, and a transparency condition in which the recorded private choices were disclosed before a final decision. The latter does not reveal the correct answer. These comparisons explore how ambiguous signals differ from neutral public text and explicit votes. However, the paper notes that differences in comment content and belief elicitation prevent the controls from establishing a distinct causal mechanism.

What changed

Across nine models, the paper reports that ambiguous public signals degraded decision quality. The largest reported drop in public accuracy was 47.8 percentage points, while collective rationality failures reached 47.3% in the most susceptible setting. These are maximum results in the tested conditions, not typical performance figures for all systems. Researchers also observed errors in the agents’ reported estimates of their peers’ private choices alongside public-stage failures.

Revealing the recorded private choices repaired many failures, although recovery varied by model. Those choices also provide task-relevant peer information, so recovery cannot be attributed solely to correction of mistaken group-belief estimates. The finding suggests that the distinction between measured private decisions and inferred group opinion matters for evaluating these systems; it does not establish that transparency will solve every real deployment problem.

The study is a controlled benchmark with designed scenarios and signals, rather than a measurement of all real-world agent interactions. The authors also caution that reported group beliefs are model outputs, not direct access to internal mental states. The work is an arXiv preprint accepted at the NeurIPS 2026 SocialAgent Workshop.