Tailored Evidence, Not Just Conversation, May Explain AI’s Effect on Conspiracy Beliefs

An AI conversation can reduce someone’s confidence in a conspiracy theory, but that does not explain why it works. In Just the Facts: How Dialogues with AI Reduce Conspiracy Beliefs, Thomas H. Costello, Gordon Pennycook and David G. Rand investigate which ingredients matter. Their arXiv preprint, submitted on November 3, 2026, reports two experiments with a combined final analytic sample of 2,193 U.S. adults.

The central finding is an asymmetry: changing several features of the exchange did not detectably weaken belief reduction, but instructing the AI to persuade without evidence or reasoned counterargument did. The authors interpret this as support for relevant counterevidence being central—not proof that every other feature is irrelevant.

Removing ingredients one at a time

Participants were recruited through CloudResearch Connect and completed the studies in English. They described a conspiracy theory they found credible and explained their reasons. An AI summarized their belief into a statement, which they rated before and immediately after the intervention on a scale from 0, definitely false, to 100, definitely true. Analyses included participants whose initial rating was at least 50 and whose responses met the study’s screening criteria.

Study 1 included 1,290 participants and used GPT-4 for the conversations. Its baseline treatment involved three rounds of exchange tailored to each participant’s belief and reasoning. Average confidence fell from 80.5 to 69.6: a 10.9-point reduction, not a 10.9-percentage-point decrease in the number of believers.

The researchers then varied individual features. They disclosed the AI’s persuasive goal, framed the exchange as a debate, instructed it to inform rather than persuade, or asked for shorter responses. Two further conditions changed when belief was rated or removed participant-specific evidence from the statement being rated. None produced a credible difference from baseline in the authors’ Bayesian comparisons.

Shortening the AI’s output roughly halved its total words across the conversation, from 1,226 to 588, without detectably reducing effectiveness. But this does not establish an optimal message length or show that information quantity never matters.

The clear exception was “No Facts.” Here, the AI was instructed to avoid evidence, alternative explanations and reason-based persuasion while still expressing disagreement. Average belief fell by only 3.8 points, substantially less than baseline. The remaining reduction also means the condition was less effective, not wholly ineffective.

One tailored message still helped

Study 2 included 903 participants and used GPT-5.2. It compared the full dialogue with a single personalized AI message and a control conversation about an unrelated topic.

Average belief fell by 14.6 points after the full dialogue, 11.8 points after the single message and 1.4 points in control. The control comparison argues against the large treatment reductions being explained simply by repeated measurement or the tendency of unusually high initial ratings to move closer to average.

The single message retained an estimated 79% of the full dialogue’s effect relative to control, with a 95% credible interval of 58% to 101%. Its estimated effect was smaller, but the comparison did not conclusively establish a difference. Back-and-forth conversation therefore was not necessary for a large immediate reduction, although it might add benefit.

Among full-dialogue participants who started above 50, 23.1% finished below 50, compared with 1.5% in control. That measures crossing from the “true” side to the “false” side of the scale—not complete rejection by everyone who changed their rating.

What the content analyses add

Post hoc analyses used language models to code persuasive strategies, individual claims and participants’ explanations. Evidence-based rebuttal and references to sources and experts were associated with larger reductions. Rapport, storytelling and empathy did not show comparable positive associations in the mediation model. Participants whose ratings fell also commonly credited facts and reasoning.

These analyses support the experimental pattern, but they do not independently prove a causal mechanism. Strategy use was measured rather than randomly assigned. Nor was a larger count of concrete empirical claims consistently associated with greater change: the results do not reduce to “more facts are always better.”

Important boundaries

The No Facts instruction removed a bundle of features, including reasoning, sourcing and specificity. It cannot isolate the contribution of facts alone. The studies could reliably detect only relatively large differences between treatment variants, so smaller contributions from warmth, framing or dialogue remain possible.

The samples were self-selected online participants, not representative samples of all conspiracy believers. Some focal beliefs were subsequently classified by another AI as mixed or plausible; analyses restricted to beliefs labeled false or mostly false retained the main pattern.

Finally, these experiments measured belief immediately afterward. The paper cites earlier research on persistence, but does not establish long-term durability for these particular variants. Its strongest conclusion is narrower: tailored, evidence-rich argumentation appears central to the immediate changes observed here.