Can an AI model tell when its own answer is wrong?

The problem: confidence is not correctness

Language models can give plausible answers even when mistaken. That makes evaluation difficult when a reference answer is unavailable. Common alternatives—asking the model how sure it is, checking how likely its words were, or sampling several answers to see whether they agree—can all mislead. A model may confidently repeat the same error.

In Circe: Training-Free LLM Self-Evaluation via Component Interaction Consistency, Shuyi Miao and colleagues propose using a model’s internal computations as additional evidence. Their central question is whether patterns in the interaction between two parts of a transformer—attention and feed-forward networks, or FFNs—can help distinguish correct from incorrect responses.

Three signals, one score

Circe combines three measurements. The first, component interaction consistency (CIC), compares the direction of attention updates in earlier layers with changes in FFN outputs induced by attention in later layers. The authors’ intuition is that correct responses may show more consistent directions across layers. CIC measures directional alignment; it does not directly verify an answer’s claims.

The second signal, representation geometry (RG), summarizes how internal representations move and change direction as information passes through the model’s layers. The third, likelihood confidence (LC), transforms the average negative log-likelihood of generated tokens into a bounded score. In plain terms, LC reflects how probable the model considered its own wording—a potentially useful clue, but not a guarantee of truth.

Circe combines these signals into one response score. It does not update model parameters or require repeated answer sampling. But “training-free” does not mean no labeled data are used at all: the researchers used development data to select domain-specific weights for combining the signals, then fixed those weights for testing. Development and test questions were separate.

What the tests found

The researchers evaluated five open-source language models on six benchmarks covering mathematics, reasoning, general knowledge, and reading comprehension. They measured how well scores ranked correct responses above incorrect ones; Circe does not independently prove an answer true.

Across the four domains, the paper reports the best average performance among the reference-free baselines on three discrimination measures. Its advantage was not universal: it achieved the highest AUROC (area under the receiver operating characteristic curve) in 18 of 20 model–domain combinations, with two exceptions. In an ablation test, the full combination performed better on average than versions omitting CIC, RG, or LC. This supports the authors’ interpretation that the signals provide complementary information in the tested settings.

The paper also reports results from additional evaluations: eleven language versions of a mathematics benchmark, a visual geometry task, and mixtures of benchmarks intended to test performance across different sources. These broaden the evidence, but do not establish reliability across all languages, images, or real-world applications.

What remains uncertain

Circe is a ranking aid, not an independent fact-checker. Its measurements come from the model being evaluated, and internal consistency or high token likelihood could still accompany a wrong answer. The experiments assess discrimination on specific benchmark responses; they do not show that using Circe in a live system reduces harm or improves user decisions.

Results may also depend on the models, tasks, response styles, and deployment conditions. The domain-specific score weights were selected using development data, and performance may differ in other settings. The multilingual and multimodal evaluations cover particular benchmarks and models, not every language or kind of visual question.

The source is an arXiv preprint; its review status is unclear from the available evidence. The findings should therefore be read as the authors’ reported results, not as settled consensus. The experiments suggest that attention–FFN interactions may add useful evidence to self-evaluation. Whether that evidence is dependable enough for consequential decisions remains an open question.