Exact Replay Does Not Guarantee Stable AI Deliberation

Repeatability and stability are different

When several language-model agents deliberate through a shared conversation, each response becomes context for whoever speaks next. A changed word or fragment can therefore influence later discussion. But a textual difference need not change anyone’s recorded preferences or the committee’s final choice.

In their preprint, Low-Margin Branch Points in Deterministic LLM Deliberation, Hajime Shimao and Warut Khern-am-nuai ask whether an exactly replayable discussion can still branch after a controlled internal intervention. Their distinction is straightforward: replay checks whether the same starting configuration produces the same transcript; a stability audit checks what happens when a specified internal state is displaced.

The authors report evidence of local susceptibility in their tested system, not a measure of ordinary deployment failures.

A committee under controlled conditions

The main experiment used Qwen2.5-7B-Instruct to run a five-agent committee discussing three health-financing policy options over 20 rounds. Each turn included free-form discussion and structured records of preferences, confidence, and tags. These records and the transcript were passed forward.

Generation used greedy decoding: at each step, the model selected the token with the highest internal score, or logit. Tokens are the pieces of text a model generates. The researchers focused on the gap between the highest-scoring token and its runner-up. A narrow gap, called a low margin, marks a choice that may be easier to redirect.

Before intervening, they required an unchanged reconstruction to reproduce the reference token sequence exactly. All 1,280 zero-perturbation rows in the main grid passed this check.

The intervention was a one-time displacement of an internal residual vector at transformer layer 27. Its size was measured relative to that vector’s original magnitude. These were deliberate stress probes, not simulations of measured hardware errors or naturally occurring numerical noise.

A branch counted when generation first differed from the reference within the ten generated tokens strictly after the intervention step. This definition includes delayed divergence, not just an immediate token switch.

Close token choices branched more often

Across 10,240 planned intervention rows, the authors report branches in 604 of 5,120 low-margin rows, or 11.8%. High-margin controls produced none in 2,560 rows. There were more low-margin rows because each target call contributed two low-margin positions and one high-margin position. The grid also included median-margin positions.

The contrast persisted after excluding the largest interventions. Removing the two largest scales left 142 branches among 3,840 low-margin rows and none among 1,920 high-margin rows.

A separate test examined intervention direction. The researchers calculated directions that locally reduced or increased the score gap between the original leading token and its runner-up. At matched positions and scales, the reported branch rates were 43.3% for margin-reducing directions and 2.5% for margin-increasing directions.

This supports the authors’ explanation that proximity to a token-choice boundary matters. It does not make the raw score margin a calibrated measure of internal distance to that boundary.

Changed text sometimes changed the outcome

The researchers continued selected branches through the remaining committee discussion and final decision. All 16 unchanged continuation replays reproduced downstream calls and the final state exactly.

Among 15 selected low-margin branch continuations, every run retained a textual difference. Four were followed by a preference or decision change, and three ended with a different final decision.

These results show that a local change can enter shared memory and affect later committee state. They do not establish how frequently that happens across all possible branches: the continuation set was selected after branches were realized. Random-direction interventions could also propagate when they produced comparable text changes.

What the audit establishes—and leaves open

Additional checks using content-bearing tokens, five further policy scenarios, a smaller 3B model, and neutral agent labels preserved the low-versus-high-margin contrast. However, they did not repeat the complete mechanism and continuation studies.

The full mechanism and continuation audit remained concentrated on one primary model, one policy scenario, one injection layer, and greedy decoding. Its main intervention rows were clustered within five target calls, rather than independently sampled deployments.

The study also does not establish decision correctness, demonstrate that committees amplify changes more than matched single-agent systems, or show that this susceptibility is unique to multi-agent AI. Its practical contribution is narrower: an audit that separates exact replay, local transcript branching, and downstream consequences rather than treating reproducibility as proof of stability.