Beyond Likely Answers: A Preprint Proposes Testing AI for Invariance

Why probability is not a truth test

Maximum likelihood training rewards a model for assigning high probability to the text in its training data. That helps it learn language and reproduce patterns of knowledge, but Lopez-Paz argues that it does not, by itself, establish whether a statement is true.

The distinction runs both ways. Rare events really happen, while false statements can become common through repetition. A model’s estimate of how likely a continuation is therefore cannot serve as a sufficient test of factual correctness.

The paper does not dismiss maximum likelihood altogether. It explicitly treats it as a valid way to acquire fluency and broad knowledge. Its objection is to treating that training objective as the final criterion for truth.

Looking for what survives change

The alternative centers on invariance: a relationship remaining stable under specified changes. Drawing on Robert Nozick, Eugene Wigner and the structural-realist tradition associated with Henri Poincaré and Ernst Cassirer, the paper argues that enduring relations and mechanisms offer a better guide to objectivity than probability within one setting.

In machine learning, the practical question is whether a system has learned a transferable rule or a shortcut specific to its training examples.

The paper illustrates this with a cow-versus-camel classifier. If most cows appear on grass and most camels on sand, a model can rely on background color while memorizing exceptions. It may classify every training image correctly yet fail on unfamiliar cows on beaches. Changing the background exposes the shortcut.

The relevant invariant is not an identical answer to every possible input. It is the rule that should continue to apply when irrelevant conditions change.

What the mathematical examples establish

The paper connects this argument to Invariant Risk Minimization, or IRM, an existing approach that seeks a representation of the data for which the same classifier is optimal across multiple training environments.

A toy regression example shows the intended distinction. A target is generated from one input, while a second input is a noisy measurement derived from that target. Using both can improve prediction within the training environments. However, the best coefficients involving the second input change with the environment’s noise scale. The coefficient using only the causal input remains stable.

This illustrates why fitting observed data well and recovering a stable generating mechanism can favor different predictors. It is not evidence that invariant learning always performs better. Lopez-Paz acknowledges that prior empirical evidence for IRM outperforming standard empirical risk minimization was limited, and that its optimization problem is computationally difficult.

A proposed language-model procedure

For language models, the paper suggests treating context as an environment: a question might be asked through different disciplinary perspectives, framings or languages.

Its proposed procedure is to:

1. Receive a question and generate relevant alternative contexts.
2. Produce an answer under each context.
3. Compare the proposals and extract their invariant content, including shared reasoning where available.
4. Abstain if no common core emerges.
5. Optionally fine-tune the model on the resulting question–answer pairs.

The intended distinction from conventional self-consistency is that the contexts change, rather than merely sampling repeatedly under one fixed context and taking a majority vote.

Reasoning traces might help identify why answers disagree—for example, whether a bird identification depends on water in the background rather than the bird itself. But the paper explicitly conditions this benefit on those traces faithfully reflecting how the model reached its answer.

The unresolved limits

Choosing which transformations are relevant is a central unsolved problem. A change that alters the substance of a question should not necessarily preserve its answer. Conversely, comparing too many contexts can leave a common answer so general that it says little.

Most importantly, different personas generated by one model are not independent sources. They may expose framing effects while repeating an error shared throughout the model’s training data. The paper points to different models, retrieval or experimentation with the world as ways to address such shared errors.

The proposed prompting procedure has no new benchmark results or measured hallucination reductions in this preprint. Its extension to continual learning—learning from stable patterns across user groups rather than demanding consistency across time—is also a suggestion, not a demonstrated result.

The contribution is therefore a research framework: test which mechanisms survive meaningful changes, while recognizing that agreement alone does not certify truth.

Source: Against All Likelihood, version 1, David Lopez-Paz. This is a preprint, not established peer-reviewed evidence.