Training AI Agents to Remember—and Check Their Memories

Two ways to remember

An assistant working across many sessions needs to recall preferences, project conventions and changing personal details. Text-based memory systems store information outside the model and retrieve relevant passages when a question arrives. Weight-based memory instead incorporates information into model parameters through training.

In Route to Remember: Adaptive Memory Selection for LLM Agents, Zeyu Fang and colleagues at George Washington University and Amazon investigate whether weight-based memory can reliably store user-specific facts. Their paper, posted on November 3, 2026, is an arXiv preprint. Its results should be read as the authors’ experimental findings, not an established demonstration of production readiness.

The central tradeoff is where the work happens: external memory requires retrieving and processing evidence when answering, while parameter-based memory requires training when information is added.

Teaching a model without giving it the notes

The researchers first extract timestamped memory statements from conversations. For weight-based memory, they turn those statements into training questions: paraphrases, questions containing false premises that need correcting, and questions about unrecorded information that should receive refusals.

Each user gets a persistent low-rank adaptation, or LoRA, adapter: a trainable addition to the base model. Their method, on-policy self-distillation, trains this student against a teacher built from the same base model, but with access to memory text.

The student generates its own answer, and the teacher supplies training guidance along that generated sequence. At evaluation, the student receives the question without retrieved memory passages. “Without context” therefore means without supplied conversation history—not without an input question.

The teacher does not see unlimited history either. The implementation caps its memory window at 12,000 characters, retaining the current session and fitting in recent earlier sessions. The adapter accumulates updates across sessions.

Stronger recall on newly answerable questions

The headline experiment uses Qwen3.5-9B and 1,438 native HaluMem questions from eight users. These questions are evaluated in the session when they first become answerable, making this a test concentrated on recently available information.

The authors report 59.9% accuracy for self-distilled weight memory, compared with 47.8% for MemOS-style retrieval, 51.9% for extracted memory supplied in context and 51.3% for raw dialogue. Ordinary supervised fine-tuning on the same augmented data reaches 36.9%.

The reported advantage over retrieval is positive for all eight users. Its user-clustered 95% confidence interval is 9.4–14.8 percentage points. Comparisons at model sizes of four billion and 27 billion parameters also favor self-distillation over the tested extracted-text and retrieval baselines.

Those findings support the importance of the training recipe within these experiments. They do not show that storing facts in weights is universally better than retrieval.

Checking answers instead of trusting confidence

The hybrid system, called verify-then-regenerate, first lets weight memory answer. It searches the text store using both the question and the generated answer. A verifier checks whether retrieved evidence supports, contradicts or fails to support the answer.

Supported answers are retained. Otherwise, the base model answers again using retrieved, timestamped evidence, without the user adapter. The text store also consolidates superseded information, helping reduce conflicts caused by outdated entries.

This hybrid reaches 66.1% accuracy on the native HaluMem questions: 6.2 percentage points above weight memory alone. It improves five of six categories, including questions that should be refused because the requested information was never recorded. However, memory-conflict accuracy falls from 76.0% to 66.1%.

The researchers’ tested confidence and query-based routing signals were much less useful for predicting correctness. Recognizing whether a question is answerable did not reliably establish whether a particular generated answer was right.

Forgetting changes the ranking

When earlier questions are repeated after additional sessions, weight memory’s advantage fades. Its accuracy falls from 59.9% initially to 32.4% after 80 sessions, while retrieval declines from 47.8% to 40.1%. Retrieval overtakes it by the 20-session measurement.

The second benchmark, LoCoMo, also qualifies the story. Weight memory scores 46.9%, barely above extracted text at 46.4%. The hybrid reaches 60.4%, largely through improved refusals on unanswerable questions; its scores decrease across the four answer-bearing categories compared with weight memory alone.

Costs and measurement limits

Weight memory processes fewer input tokens when answering, but that is not equivalent to lower total operating cost. Training and question generation add upfront work. In the reported evaluation, weight memory uses 1.0 GPU-hour for reading, versus 0.9 for retrieval; the hybrid uses 1.8. The token accounting excludes the distillation computation itself.

Evaluation is another limitation. Claude Sonnet 4.6 judges answers without partial credit, and the authors identify inconsistent judgments and inadequately supported reference answers. Some explanatory passages also conflict with the tables: the reported HaluMem hybrid improvement over the best text baseline is 14.2 percentage points, not the prose’s 18.2.

The practical finding is therefore conditional: trained adapters can provide competitive recall for recent facts, while an external evidence store remains valuable for older information, updates and checking unsupported answers.