When AI Conversations Go Off Track, Rebuilding Context Can Help

A chatbot’s earlier answers can become a liability when they preserve assumptions the user has already corrected. Researchers from the University of California, San Diego, and Microsoft Research investigate how to remove that misleading history without losing the information a continuing conversation needs.

Why remembering everything can backfire

In the preprint Agentic Context Management for Multi-Turn Human–AI Conversations, Matthew Ho and colleagues examine a failure mode called context pollution. An assistant makes an early mistake about what the user wants, records that interpretation in its answer, and then continues to rely on it—even after later messages supply a correction.

One proposed remedy is assistant omission: remove every earlier assistant message before generating the next response. That can work when the user’s messages collectively contain the complete task. But it breaks down when someone asks to revise an earlier draft or continue from a diagnostic result. In those cases, deleting the assistant’s history also deletes information needed to answer.

The authors distinguish these referential conversations from self-contained tasks. They reserve “stateful” for cases where the missing information concerns the environment, such as results returned by troubleshooting tools.

A separate reviewer rebuilds the task

Their method, Agentic Conversation Context Curation, or AC3, operates before the assistant responds and requires no additional model training.

First, an analyzer constructs an up-to-date task specification from user messages and the system instructions. Earlier assistant messages are physically excluded from this stage. Second, the analyzer receives that specification alongside the full conversation and checks which parts of the assistant’s work agree with it.

The system then applies one of three interventions:

Augment adds the analysis to the existing conversation.
Reset replaces the history with a fixed template containing the specification and work the analyzer judged correct, followed by the latest user message.
Rewrite uses another model call to restructure the context.

Reset and Rewrite act only when the analyzer flags issues. The default, Gated Reset, also delays intervention and limits resets to three per conversation. In tool-use settings, the analyzer additionally records observed environment state.

The authors report that shielding the specification extractor matters. In single-trial ablations, allowing it to see assistant messages reduced math accuracy from 80% to 40% and code accuracy from 53% to 11%. Instructions to disregard assistant-originated assumptions did not close the gap. Their interpretation is that a reviewer can inherit the same misleading assumptions it is supposed to identify.

Where the method helped

The strongest headline results come from Lost in Conversation, which splits complete questions into information revealed over several turns. Because the user messages contain the task, assistant omission is a particularly strong baseline by design—not a guaranteed accuracy ceiling.

On difficulty-selected instances, Gated Reset closed 55–84% of the gap between full context and assistant omission across math, code, and function-calling tasks, and exceeded omission on database queries. These experiments mostly regenerated the final response from a fixed conversation history rather than testing an entirely new conversation from start to finish.

The authors also report end-to-end results on complete, unselected database and code pools. With a GPT-5.4-mini assistant, Reset reached 75.7% database accuracy versus 56.1% for full context, and 92.0% code accuracy versus 83.0%. Faithful history summarizers did not reproduce those gains, including a two-call version that used more compute than Reset.

On real WildChat conversation prefixes, a GPT-5-mini judge preferred the reported AC3 responses over assistant omission and full context. For GPT-5-mini responses, Gated Reset won 86.1% of comparisons against omission and 83.8% against full context. Those are model-judge preferences, not demonstrated human preferences: separate checks with other judges sometimes favored omission instead.

What rebuilding still loses

Results were mixed on CollabLLM, a simulated collaborative conversation benchmark. In the telecom troubleshooting subset of tau2-bench, assistant omission scored zero for every tested respondent because removing diagnostic history caused repeated lookups until the step budget expired. AC3 avoided that collapse, but no AC3 operator outperformed full context. Gated Reset remained roughly 7–12 percentage points behind it across three respondents.

The authors also explicitly reject a surgical-editing interpretation. In an injection study, Reset removed false information 97.6% of the time, but preserved the injected true information only 4.0% of the time. Its benefit comes from discarding assistant history and rebuilding the specification, not reliably retaining useful individual passages.

AC3 adds two to three model calls per analyzed turn. Many reported results are single trials; long-horizon tasks and broader agentic domains remain untested. The preprint therefore supports context rebuilding as a useful intervention in some settings, not a universal replacement for conversation history.