WASP-Mem preprint tests wiki-style memory for workplace AI agents

Workplace automation can depend on details missing from an AI agent’s initial instructions: organization-specific rules, undocumented service behavior or recovery steps after an unsuccessful action. The WASP-Mem preprint investigates whether agents can acquire that knowledge through persistent memory without changing the underlying model’s parameters.

Its central finding is qualified but practical. In the reported experiments, freely organized procedural wiki pages generally supported better execution than raw records or the structured alternatives tested. Human explanations of failures also helped more than repeated reliance on the agent’s own assessment. These are benchmark findings, not evidence that the approach is ready for unrestricted enterprise deployment.

Turning experience into procedures

WASP-Mem stands for Wiki-based Agent-Synthesized Procedural Memory. Rather than simply preserving a transcript, it asks a consolidator agent to extract reusable guidance: when a procedure applies, how to carry it out, what can go wrong and how to recover.

The system starts with an empty procedural wiki and an index file. After execution, an agent reviews traces and available feedback, then creates or revises interconnected pages. Later agents retrieve relevant material using semantic search and navigation through the wiki. The memory remains frozen during each execution batch; updates become available in the next round.

The basic “Free Form” version imposes minimal page metadata but lets the agent decide the remaining organization. Two alternatives add prescribed page types and fields, with one also using explicit rules to identify pages for merging. Free Form still has navigation and quality checks: it is not an absence of system design.

Better checkpoint scores, with important distinctions

The primary benchmark, TheAgentCompany, contains 127 workplace tasks and 542 execution-based checkpoints across GitLab, RocketChat, OwnCloud and Plane. The researchers tested Claude Opus 4.6, GPT-5.4 and GLM-5.

In the single-round comparison using checkpoint feedback and evidence from both successes and failures, Free Form achieved checkpoint pass rates of 72.3–72.5%. The corresponding agents without memory scored 58.5–60.0%. The Free Form stores occupied 233–255 KB, compared with roughly 8,000–8,500 KB for raw-trace storage.

Checkpoint success measures partial progress, not necessarily completed tasks. Free Form did not win every end-to-end comparison: for Claude on TheAgentCompany, the schema-based variant fully passed 48.8% of tasks, versus Free Form’s 48.0%. The authors caution that small differences remain unresolved without replication.

The researchers also tested a modified retail domain of τ²-Bench, withholding a custom operating procedure from the agents. Here, 74 training tasks and 40 test tasks were disjoint. Free Form led or tied checkpoint performance among the compared methods for each model. This provides evidence of transfer to unseen tasks within that setting, distinct from the repeated workloads of the primary experiment.

Why diagnosing failures mattered

The feedback experiments separated self-assessment, external pass/fail outcomes and checkpoint outcomes supplemented by human diagnosis.

With both successful and failed evidence included, one round of Free Form memory reached 64.4–64.9% checkpoint success using self-assessment, 72.3–72.5% using checkpoint outcomes, and 75.8–77.1% with checkpoint outcomes plus human explanations. The diagnostic condition therefore added 3.5–4.6 percentage points over checkpoint outcomes alone, according to the detailed results.

Failure evidence was not automatically helpful. With self-assessment, adding it produced little benefit or a decline. With external outcomes and human diagnosis, including failures rather than only successful checkpoint evidence improved checkpoint scores by 9.6–10.5 percentage points.

After four rounds with diagnostic feedback, checkpoint success reached 85.6–86.7%. One diagnostically supervised round already exceeded four self-assessment rounds by 7.0–10.9 percentage points. That comparison combines external outcome information with human explanations; it should not be read as isolating human diagnosis alone.

What the study does not establish

TheAgentCompany training and evaluation used the same tasks, with the authors reporting post-consolidation guardrails against case-specific leakage. Its repeated-round results measure improving competence on recurring workloads, not unrestricted generalization. Each configuration ran once, leaving variability unquantified.

Human feedback also required approximately six annotator-hours per 127-task rollout batch. Longer-term memory stability and scalable automated diagnosis remain open questions.

The authors’ proposed division of labor is therefore a design hypothesis supported within these experiments: provide a general memory system, let agents organize the procedures, and direct human effort toward explaining observed failures. The results do not establish that every schema or explicit deduplication method is harmful.