Noisy scores shape AI search—but a temperature model misses its outcomes

A temperature for selection

A language model proposes a revised program or prompt, an evaluator scores it, and the search keeps it if it beats the current candidate. Noise complicates this simple loop: a worse candidate can win a comparison, while a better one can lose.

In a November 2026 arXiv preprint, Xiaomeng Li connects this uncertainty to an effective temperature. More noise weakens selection; less noise makes comparisons more decisive. The idea builds on established work in statistical mechanics, evolutionary models and probabilistic choice.

The mathematical result has strict conditions. When both candidates receive fresh evaluations, their noise difference follows a logistic distribution, and the proposal process is reversible—satisfying a balance between forward and reverse moves—the search has a particular invariant distribution. It reweights the equilibrium of mutation without selection, giving higher-quality candidates exponentially more weight. This is the paper’s “tilt.” Invariance alone does not guarantee convergence from a given start.

For other continuous symmetric noise differences, the corresponding result concerns a limiting process with increasingly small proposal steps, not arbitrary edits. Under an approximate minibatch model, inverse temperature grows with the square root of batch size. Real paired comparisons can violate the approximation through ties, asymmetry and candidate-dependent noise.

Predictions committed before search

Li tested the tilt using language models that generated Python heuristics for online bin packing and travelling-salesman tours. Seven settings combined two Claude proposer models and instructions requesting neutral edits, small changes or substantial rewrites.

For each setting, calibration included 24 walks of 100 steps that accepted every proposal, without selection. Samples from the walks estimated the mutation equilibrium; parent–child score differences estimated evaluation noise. Predictions were committed to version control before independent search runs began.

Search compared candidates on fresh shared minibatches of 1, 4, 16 or 64 instances, with six seeds per batch size. The main outcome averaged the incumbent’s quality over steps 100–150. Quality meant performance across the same fixed 200-instance pool supplying those minibatches—not generalisation to unseen problems. Larger batches also cost more per step, so these comparisons do not establish equal-cost efficiency.

Bigger batches helped, but predicted levels failed

The measured level rose with batch size in six of seven settings. Li treats this as a basic check: many explanations of noisy selection predict that direction.

The numerical predictions were less successful. Across bin-packing settings, aggregated results for editing proposers reached 62–88% of the predicted gain above unselected mutation. Rewrite proposers reached 109–119%. These percentages describe gains relative to calibration, not absolute task accuracy; individual batch-size results varied more widely.

The travelling-salesman results could not provide a sharp test. Runs varied substantially, and at larger batches the weighted calibration estimate effectively depended on only one or two samples.

An initially observed bin-packing pattern linked stronger parent–child quality correlation to a smaller attained fraction of the predicted gain. Registered tests on two additional settings supported that direction. However, correcting uncertainty calculations for proposals shared across batch sizes removed one quantitative pass. With calibration uncertainty included, agreement with the fitted relationship remained unresolved.

Starting history still mattered

Some searches began from a strong program rather than the baseline. Small-edit runs lost quality and remained below predictions after continuation to 300 steps, contradicting one specific reversible model of slow convergence—not reversibility in general.

A further registered experiment compared twelve similarly good starting programs: six previously selected by search and six visited by unselected walks. Its primary early-window comparison was inconclusive. A secondary window showed greater losses from search-selected starts.

After 300 steps, the groups remained separated, but only narrowly under the registered rule. The result was sensitive to omitting individual starts, origins were not randomised, and two seeds shared across starts were not independent replications. The continuation was rerun after an evaluation-environment fault was repaired. Overall, different starts had not been shown to reach a common stationary level.

Prompt optimisation broke the pattern

A registered transfer test searched system prompts for a synthetic letter-counting task. At batch size four, small-edit search reached a frozen score of 0.756 against a tilt prediction of 0.474; bolder revisions reached 0.781 against 0.674. The correlation relationship went in the opposite direction from bin packing.

Each prompt evaluation sampled task-model answers once, producing a frozen per-instance score vector. The shared starting prompt was evaluated independently once per setting. These frozen empirical scores were not expected accuracies under repeated sampling: search could select lucky evaluations as well as better prompts. The small-edit calibration walks were still drifting. Short calibration was a proposed explanation for the underestimate, not a tested resolution.

Caching changes the problem

Keeping an incumbent’s old score introduces another issue: selection preserves lucky overestimates. An exploratory bin-packing experiment found cached scores inflated relative to full-pool quality, without cached search outperforming fresh comparisons at matched evaluation budgets. Simulations favoured caching instead, so neither ordering is universal.

The practical lesson is diagnostic, not a guaranteed ceiling: check score inflation, calibration drift and sensitivity to starting conditions. A search that appears flat has not necessarily reached equilibrium, and this study does not establish the tilt as the stationary law of language-model editing.