Reglos separates learning a user’s preferences from improving how AI uses memory

A personal assistant can remember a conversation and still give the wrong answer. It might apply an outdated preference, mistake a relative’s tastes for the user’s, or repeat something the user asked it to forget. In a preprint, Sen Yang, Jiaduo Lin, Zehong Huang and Wen Liu present Reglos, a system designed to address both what an assistant stores and how it uses that information.

The distinction matters because better retrieval is not always the missing ingredient. In an audit of 159 development failures, the authors attributed 31% to cases where the necessary evidence was already in the model’s context, but the model chose incorrectly. Another 48% required facts never stated in the conversation. Those findings concern the audited cases, not a universal breakdown of memory-system errors.

Two loops, different responsibilities

Reglos separates improvement into two processes. An online loop updates each user’s memory as conversations arrive. An offline loop changes the “harness”: the software and instructions that turn stored information into context for a language model. The underlying task model remains frozen, so “self-improvement” here does not mean that the model retrains itself.

The memory record preserves a fact’s wording, its source messages, when it applies and when the system learned it. If a preference changes, the old fact’s validity ends without removing its history. A nightly process called Dream summarizes sessions and rebuilds the user’s profile from current facts.

Forgetting works differently from ordinary deletion. A request expires matching facts and creates a record of the request, telling subsequent processing not to rely on that information. The automatic pipeline retains historical rows; users can separately delete memories through the app. This is a mechanism for controlling use, not a claim that a conversational request physically erases all stored information.

Retrieval combines searches over meaning, keywords and relationships between entities, with time, location and recency signals. The harness presents retrieved evidence alongside instructions about outdated preferences, forgotten facts, sensitive identifiers and preferences belonging to other people.

Improving the software without rewarding shortcuts

The offline process uses a coding agent to inspect failure traces and propose patches. A critic from another model family independently examines failures and screens proposals for benchmark-specific shortcuts, such as logic tied to answer templates or labels.

The timing of the safeguards is important. The reported harness was built through development-stage changes and then frozen. Only afterward did the authors adopt a stricter acceptance rule for further patches: improve on both validation groups containing different personas, while costing a second answering model no more than one percentage point. Personas inspected during development were excluded from that evaluation pool. The earlier development changes were not all selected under this stricter rule.

The authors report that four harness changes raised validation accuracy from 54.2% to 60.7%. Changes included clearer memory-use instructions and retrieval queries built from the words distinguishing multiple-choice options. These queries aim to uncover relevant evidence that the question itself does not name.

What the benchmarks show

With Qwen3.8-27B writing memory for every system and also answering questions, Reglos reached 61.3% on PersonaMem-v2 and 84.0% on LongMemEval-S. These were respectively 7.5 and 3.6 percentage points above the strongest comparison memory systems in their standard configurations. With GPT-5.6 Luna answering from the memories, Reglos scored 57.5% and 87.2%.

Controls suggest that instructions explain part, but not all, of the advantage. Giving EverOS the Reglos memory-use policy raised its PersonaMem-v2 score to 58.9%, leaving a 2.5-point gap. Raw conversation retrieval inside the Reglos harness reached 58.1%, compared with Reglos’s 61.3%.

The gains were uneven. On PersonaMem-v2, Reglos’s strongest advantages concerned sensitive information and requests to forget. EverOS led on four of five ordinary preference categories. Preferences belonging to someone other than the user remained below chance for every system, exposing an unresolved attribution problem.

Limits behind the headline

LoCoMo does not isolate Reglos’s memory retrieval: its histories were short enough for the system to pass the full dated conversation to the reader. With GPT-5.6 Luna, the full-context baseline also slightly exceeded Reglos. On BEAM, results were mixed rather than a clean sweep.

Comparison conditions matter too. One model wrote memory throughout, and the cited best published PersonaMem-v2 result used a different reader. Some baseline configurations omitted an opening persona description; the authors tested variants adding it.

Training-free also does not mean inexpensive. The authors estimate roughly 50 million task-model tokens and six hours to construct memories for 200 test personas. The preprint supports a narrower conclusion than unlimited autonomous improvement: explicit memory states and carefully revised evidence presentation can improve benchmark performance, while attribution errors and reader decisions remain substantial obstacles.