Where Does an AI Agent’s Learning Persist?

Looking beyond the model

Knowledge distillation uses a teacher-student framework to transfer capabilities at reduced cost. It has been widely studied for large language models (LLMs). But agents can do more than generate a response: they may solve tasks through sustained, closed-loop activity, where actions and subsequent results feed into the ongoing process. The authors argue that studying transfer only at the model level misses other possible places for knowledge to persist.

In A Comprehensive Survey of Agent Distillation across Models, Artifacts, and Harnesses, the authors define agent distillation as persistent knowledge transfer between teacher and student agents. They distinguish experience used during transfer from knowledge retained for reuse, then organize knowledge by its substrate—the category in which it persists: models, artifacts, or harnesses.

The abstract names these categories but does not define their contents in detail. It therefore supports a broad map, not specific claims about what belongs in each category or how a particular transfer method works.

A framework, not a contest of methods

The survey says it analyzes transfer paths across these substrates, including hybrid strategies that combine them. It also reviews benchmarks and evaluation protocols addressing three questions: whether transferred knowledge is retained; whether an improvement can be causally attributed to the transfer; and whether the agent remains useful in deployment without further teacher support.

These are the survey’s topics, not reported findings that particular approaches succeed on those tests. The abstract provides no benchmark scores, head-to-head comparisons, or evidence that one substrate is more effective than another. Its contribution, as presented there, is a unified way to organize and examine agent knowledge transfer—not a demonstration that agents can continually improve or that any strategy is best.

What the abstract leaves open

The authors say existing surveys have largely examined model distillation and agent-system design separately, limiting comparison of transfer strategies across agent components. Their proposed taxonomy aims to provide a shared basis for such comparisons and for developing efficient, adaptable agents.

This account is limited to the paper’s abstract. The full survey’s detailed taxonomy, review methods, cited evidence, and analysis cannot be assessed from the supplied material. The source is an arXiv preprint; the evidence provided does not establish peer review. For now, the framework offers useful questions: what knowledge is transferred, where does it persist, and can the student use it without the teacher? The abstract alone cannot answer which approaches perform best.