What an AI loses when it forgets the conversation

An assistant offers a shuttle, and the user accepts that offer. If the assistant later remembers only the acceptance, what has it actually retained? The decision survives, but the information needed to interpret it may not.

That dependency sits at the heart of a preprint by Danny Kim and Fernando Koch. Their study examines two different ways to reduce conversational history: supplying fewer retrieved passages and selectively keeping messages. It finds measurable token savings, but neither a universal memory budget nor proof that the resulting benchmark scores measure answer correctness.

Fewer passages, lower assigned scores

The first experiment used 1,100 questions from BEAM, a benchmark containing synthetic conversation histories, human-validated questions and scoring rubrics. Of these questions, 990 were labeled answerable and 110 required abstention: withholding an answer when information was unavailable.

The researchers held the retrieval ranking and answering model constant. A keyword-based retrieval method ranked passages from each history, and Qwen2.5-14B-Instruct received the first one, three, five, ten or twenty passages. This design compared different amounts of retrieved evidence rather than different retrieval systems.

Moving from ten passages to three reduced reader-token use by 67.78%. Tokens are the units of text processed or generated by the model; here, the accounting included both input and output, but excluded retrieval, preparation and judging.

The saving came with a 5.45-point reduction in mean assigned score on a 0–100 scale. Among the 1,084 questions with valid scores in both conditions, the averages were 43.43 for three passages and 48.88 for ten. The estimated loss had a 95% interval of 3.35–7.59 points, calculated by resampling source groups within this fixed cohort, not by repeating model runs.

Increasing the budget from ten passages to twenty used 96.24% more reader tokens for a 2.39-point score gain. At the upper tested budgets, additional tokens therefore bought smaller score gains. That does not identify an optimal budget, particularly because the scores have an important limitation.

A benchmark score is not verified correctness

The original automated judge received a scoring criterion and the candidate answer, but not the question or full conversation. Its scores measured rubric credit, not the percentage of answers independently established as correct.

The researchers audited a selected sample with an AI reviewer. That review did not resolve source-grounded correctness: it knew the hypothesis and aggregate results, searched selected portions of histories and lacked an exact recorded model revision. It was not the independent, outcome-blind human review originally planned.

A proposed replacement judge, designed to use source evidence, also failed its qualification gate. It passed every required check on only 22 of 31 synthetic cases, so the researchers did not use it to rescore BEAM.

These limitations matter when interpreting refusals. Smaller budgets generally lowered scores for answerable questions, while pooled abstention scores were higher than at ten passages at every smaller tested budget: approximately 10.9 points higher at five passages, 11.4 at three and 9.1 at one. The abstention gains therefore did not increase monotonically as the budget shrank. Nor do those gains establish that the model became better at refusing unsupported answers. Without trustworthy source-aware evaluation, higher refusal scores and more correct refusals remain different claims.

Keeping the user can lose the meaning

The second experiment made the required evidence explicit. Controlled conversations tracked three values: item, place and duration. User requirements persisted until changed; an assistant's offer changed nothing unless the user accepted its named slot.

Paired versions differed at a crucial message. In one, the user explicitly assigned a value. In the other, the user accepted an earlier offer containing that value.

One retention policy kept user messages plus the last three messages. Its comparison kept a recent, whole-message suffix at a matched prompt-token count. User-history retention preserved all necessary state information in every direct-assignment case, but none of the reference-dependent cases, by design. Matched recency preserved complete evidence in 20 of 64 cases of each kind.

Qwen reconstructed the full state in 52 of 64 direct cases under user-history retention, compared with 13 under matched recency. For reference-dependent cases, retention produced only two completed full-state matches, with one additional response missing, versus 13 matches under recency.

An appropriate null response to a missing reference could still fail the full-state metric. Conversely, guessing the original value correctly did not establish that the retained text supported it.

Preserve dependencies, then validate answers

Mistral-7B performed near the floor across retention conditions, including full history. The cause remained unexplained, limiting generalization beyond Qwen. The controlled tasks also lacked held-out confirmation.

The preprint's practical lesson is narrower than “remember less.” Direct requirements and accepted offers need different evidence. A useful memory policy must preserve those dependencies, while its evaluation must distinguish supported answers from lucky guesses and benchmark credit from correctness.