When better predictions can mean worse causal answers

A model can become better at predicting recorded outcomes while becoming worse at predicting the effects of an intervention. In a theoretical preprint posted on November 2, 2026, Marcus A. Thomas develops mathematical conditions under which these objectives conflict. The work presents proofs and constructed examples, not an empirical evaluation of deployed AI systems; it reports no datasets generated or analyzed.

Its central claim is conditional, not a blanket assertion that learning causal relationships is impossible. The paper asks when useful predictive dependencies violate the causal constraints a computation is supposed to respect, and what correcting those violations requires.

A useful signal can give the wrong answer to a change

One example separates an outcome’s direct cause from an informative but noncausal signal. The outcome combines a contribution from a parent variable with an unobserved influence. A second measured variable contains that same unobserved influence plus independent noise.

That second variable helps predict the outcome. But intervening to replace its generating equation leaves the outcome unchanged in the example’s causal structure.

A predictor nevertheless benefits observationally from using it. Under the specified assumptions—independent, centered underlying variables with unit variance—the optimal coefficient on this extra signal is one-half. Starting from a coefficient of zero, sufficiently small gradient-descent steps improve prediction while introducing a false response to interventions on the signal.

The distinction is between exploiting information and representing an effect: a variable can help reveal what happened without being something whose manipulation changes the outcome.

When the trade-off cannot be optimized away

The paper extends this example with a result for deterministic predictors with finite second moments, evaluated using squared-error loss. Its benchmark is the best predictor using only the outcome’s direct causal parents.

Any positive observational advantage over that benchmark requires dependence on additional inputs. The result does not depend on which learning algorithm produced the predictor.

Turning that dependence into an intervention-error bound requires further assumptions. The compared changes must be admissible interventions that preserve the worldly outcome. The paper constructs a comparison by drawing two extra-input values independently from their observational conditional distribution at fixed parent values. Under these conditions, retaining a specified positive predictive gain entails a positive lower bound on root-mean-square error in the predicted response.

This is not a guarantee about arbitrary interventions outside the observed support. Nor does it rule out separating observational prediction from intervention prediction or revising the task representation. Those changes alter the setup, and their causal adequacy still needs assessment.

Shared parameters create another route to damage

A separate result considers a mechanism that initially gives correct outputs but shares parameters with other predictions. An update driven by improving those other predictions can disturb the correct mechanism while reducing total observational loss.

Parameter sharing alone does not establish this outcome. The update must actually change the assessed mechanism’s outputs, alongside the stated smoothness and baseline conditions, including a zero gradient for that mechanism’s own loss. Other results concerning growing sensitivity to intervention implementations require additional sensitivity-growth assumptions. These are conditional analyses of learning dynamics, not findings that every training update worsens causal reasoning.

The same intervention needs a coherent answer

Thomas also distinguishes different ways of realizing an intervention. If two physical processes set the same target in the same relevant context, they need not automatically produce the same worldly outcome. Treating them as equivalent requires a substantive assumption; otherwise, the intervention description must specify its implementation.

On the model side, structural alignment requires outputs to agree across eligible realizations when the system’s randomness is held fixed. Matching average answers, or even entire output distributions, is weaker. The paper illustrates this with outputs U and −U, where U takes only the values +1 and −1 with equal probability. Both outputs have identical distributions, but they disagree at every possible fixed realization of U.

Alignment is also separate from correctness: a model can give the same wrong answer consistently.

Experiments cannot supply missing hypotheses by elimination alone

Another limit concerns the causal alternatives actually admitted within an inquiry, rather than everything a network could theoretically represent. If predictions remain restricted to alternatives with a positive best-achievable approximation error, neither observational nor experimental evidence can push error below that limit without changing the available alternatives.

Experiments can resolve disagreements within a suitable class. With finitely many candidates, the true model included, pairwise distinguishable intervention predictions, and exact observation of intervention distributions, successive tests can identify the survivor. These assumptions exclude noisy experimental estimation.

The paper ends with a conjecture: reliably acquiring and maintaining exact interventional competence when existing commitments fail requires autonomous participation in proposing, testing, and revising causal alternatives. The preceding theorems do not establish that conjecture. A proposed language-model arithmetic experiment is likewise a testing protocol, not a reported result. The demonstrated limits concern specified predictive setups and hypothesis classes; the broader account of autonomous correction remains a proposal.