SumOne Trains a Small Model to Condense AI Agent Histories

AI agents that search the web or work on code need to remember what they have already done. But repeatedly feeding an expanding interaction history to a language model consumes tokens and eventually runs into context limits. Discarding old material is cheaper, yet it can also erase a constraint, a useful finding or the result of a failed test.

A preprint by Jizhe Zhang, Ang Li, Chenheng Zhang and Yisen Wang proposes a separate model called SumOne to handle that trade-off. Rather than training it on examples of ideal summaries or rewarding it directly for success on a particular task, the researchers train it to preserve information that helps another model reconstruct missing parts of an agent’s history.

In the paper’s main experiments, SumOne reduced estimated inference cost substantially compared with retaining the full available history, although both search and coding performance declined. The results also suggest that the same trained summarizer can be reused with several agent models and software frameworks, within the settings tested.

Learning from missing history

SumOne starts with an older section of an agent’s interaction history and produces a shorter textual memory. During training, the researchers divide that history into blocks and randomly mask some of them. A separate, frozen language model then tries to recover the missing blocks using the summary, the visible portions of the history and retained context.

Another language model judges the reconstructions against the original text, assessing both whether the recovered content is supported and how much original information was recovered. The training reward combines those judgments with a penalty for longer summaries. A reinforcement-learning method called GRPO uses the reward to improve the summarizer.

The idea is that a useful summary should make missing information easier to recover without simply reproducing the entire history. Reconstruction is a proxy for information preservation, not a direct measurement of whether every detail will matter to an agent’s eventual task.

The researchers built approximately 5,000 training instances, roughly balanced between web-search and software-engineering trajectories. They initialized the summarizer from a 2-billion-parameter Qwen model, used a larger frozen model for reconstruction and masked half of the history blocks. At deployment, only the summarizer is retained; the reconstruction and judging models are not needed.

Lower costs, with a performance trade-off

The main comparison used the Qwen3.5-35B-A3B downstream model with the OpenCode harness. Compression methods operated at a 32,000-token threshold, while the full-history baseline retained the trajectory up to the downstream model’s native context limit.

On DeepSearchQA, SumOne scored 0.48 F1, compared with 0.52 for full history. Cumulative agent input fell from 2.83 million to 591,000 tokens per query, and estimated total cost fell from $0.170 to $0.038—about 78%. The agent summarizing its own history scored 0.44 at an estimated $0.044 per query.

On SWE-bench Verified, SumOne resolved 59% of tasks, compared with 63% using full history. Agent input fell from 4.1 million to 1.09 million tokens per task, and estimated cost from $0.286 to $0.071—about 75%. Self-summarization resolved 56% at $0.082 per task. A separate learned compressor, CoMem, cost less than SumOne in this coding comparison ($0.065 versus $0.071), but achieved a lower resolved rate of 57%.

These costs are estimates based on the paper’s listed API prices, without cache-hit discounts or other temporary pricing adjustments. One passage in the preprint describes the SWE-bench cost as 58% of full-history cost, but the table’s $0.071 and $0.286 figures imply approximately 25%.

How far does the memory transfer?

The authors also tested the same SumOne checkpoint with four additional downstream models and across four agent harnesses without extra fine-tuning. On the reported cross-model search and coding tests, it outperformed self-summarization. In the cross-harness search test, it beat self-summarization in three harnesses and tied it in Codex; it did not exceed full-history scores in those four comparisons.

On the separate LongMemEval-S memory benchmark, SumOne scored 0.47 overall, ahead of the tested compression methods Prompt-2B at 0.43 and CoMem at 0.31, but below full history at 0.77. It was not strongest in every memory category: Prompt-2B scored higher on multi-session questions, and CoMem on knowledge updates.

The study leaves important limits. Training data centered on search and coding, so other domains may need different kinds of memory. Training quality also depends on the frozen reconstructor and the model judging its output. The authors say code and implementation resources will be released upon acceptance. For now, the preprint offers evidence of a useful cost–performance trade-off under the reported conditions, not a guarantee that compressed histories will retain everything an agent may need.