When Favoring Unusual Answers Can Help Experts Tell the Truth

A contest over credibility

In the preprint Separating the Wheat from the Chaff, Johannes Hörner and Paula Onuchic study a mathematical contest between two speakers and a judge. One speaker knows the relevant fact; the other, called the “quack,” has no information about it. Both want to be selected. The judge wants to choose the informed speaker but initially considers either equally likely to be the expert.

The speakers submit their reports simultaneously. The judge also receives a noisy private signal about the truth. In the benchmark model, the unknown state is uniformly distributed between minus one and one, and the signal’s error is uniformly distributed within a bounded range.

These assumptions let the judge check whether a report is consistent with her information. A truthful expert always passes. An uninformed speaker may pass by luck.

The research uses equilibrium analysis and mathematical proofs. Its applications to chatbots and media are interpretations of the model, not experimental demonstrations of their behavior.

Why the tie-breaking rule matters

The central result concerns the equilibrium that gives the judge the highest probability of selecting the expert. In this equilibrium, the expert reports truthfully. The uninformed speaker draws reports from exactly the same distribution as truthful answers, although those reports are independent of the actual state.

When only one report is consistent with the judge’s signal, she identifies the expert. When both are consistent, neither speaker is more likely to be informed. Her choice in these ties is crucial.

Simply choosing randomly would encourage the quack to concentrate on moderate reports, which are more likely to pass the consistency check. Instead, the authors construct selection rules that favor more extreme reports when both reports are consistent and equally credible. This compensates for extreme reports being less likely to pass in the first place, making every report equally attractive to the uninformed speaker.

Here, “extreme” means closer to an endpoint of the state interval and less likely, before the signal is observed, to appear plausible. It does not mean politically radical or inflammatory. The judge never needs to choose a report she considers less credible.

The authors prove that these incentives also make truthful reporting optimal for the expert. Under their specified “max” and “min” selection rules, it is uniquely optimal. The advantage for unusual answers therefore supports truthfulness rather than inducing exaggeration.

What changes outside the benchmark

The uninformed speaker’s behavior depends on the environment. With a sufficiently thin-tailed, symmetric state distribution, the quack avoids sufficiently extreme reports and imitates the truthful distribution within a central interval.

Initial reputation also matters. When the judge initially favors one speaker, that speaker, if uninformed, uses a more moderate reporting strategy in the constructed equilibrium. The initially disfavored quack continues to use the full uniform distribution.

A further extension allows both speakers to have imperfect information. Truthful reporting remains sustainable when the less-informed speaker observes a suitably regular garbling of the better-informed speaker’s signal. Here, truthfulness means accurately reporting that signal—not necessarily the underlying state. The nested information structure matters: the proof does not directly extend to speakers who observe independent noisy signals.

Better knowledge helps—but learning has several measures

In the benchmark truthful equilibrium, judges with more precise private information identify the expert and learn the state with certainty more often. Applied to media or AI, this suggests a mechanism through which access to identical sources can widen differences in people’s ability to identify the truth exactly.

But the paper also establishes an important qualification. Less-informed judges gain a larger absolute reduction in expected uncertainty, measured by posterior variance. On that measure, communication narrows informational differences. Whether the same sources increase information inequality therefore depends on what counts as learning.

Implications within clear boundaries

The authors interpret the informed speaker as a grounded chatbot and the quack as a system lacking the information needed for a query. Their model predicts conditions under which comparative rewards sustain truthful grounded responses, while ungrounded answers become more or less accommodating to evaluators’ expectations.

This is an equilibrium possibility, not a guarantee that ordinary human-feedback training will discover the required selection behavior. The model also admits uninformative equilibria.

Within the specified environment, the truthful equilibrium reaches the judge’s best feasible expert-selection performance; more elaborate communication mechanisms cannot improve it. That conclusion retains the model’s constraints: exactly two speakers with different information roles, selection-based rewards, no transfers, and a judge who must choose someone. It is not a general prescription to reward surprising answers or a proof that real-world debate formats are interchangeable.