Turning AI adapters back into readable instructions

Can a learned skill become a document again?

A skill document tells an AI assistant how to handle a particular class of requests, from reviewing code to following software conventions. Putting that document into every prompt uses valuable context. One alternative is to encode its influence in a low-rank adapter, or LoRA: a relatively small collection of extra numerical weights that changes a model’s behavior.

The trade-off is readability. People can inspect and revise a document, whereas an adapter is harder to understand and generally tied to its original model.

In a preprint posted on November 6, researchers Saranya Dintakurthy Swadha, Pavan C Shekar, Arpit Jain and Aswanth Krishnan of QpiAI in Bengaluru investigate the reverse journey: producing a skill document from an adapter and its base model. Their results suggest this can work, but not as a general-purpose recovery of everything an adapter has learned.

Running the model beats reading the numbers

The team compared three approaches. One trained a system to translate adapter weights into inputs for a document writer. Another identified vocabulary associated with the weights and retrieved potentially relevant instruction lines. The third, called the Executing Writer, attached the adapter to its original model and added a small trainable writer adapter. The base model and the adapter being read remained unchanged.

The distinction is practical: instead of learning to interpret millions of numbers directly, the Executing Writer uses the adapted network’s own computations to produce text.

On 37 held-out skills, its two variants recovered about 39–44% of the original documents’ key instructions according to the primary AI judge. A writer using keywords and retrieved lines recovered about 22%; retrieving a similar training document scored about 27%.

Replacing the correct adapter with another skill’s adapter reduced the Executing Writer’s coverage to roughly 1–2%. That control supports the conclusion that its output depended on the adapter, rather than merely producing plausible instructions.

Nevertheless, approximately 39% of the instruction units it generated were unsupported by the original document. Better than the alternatives did not mean reliably faithful.

Stored text is easier than learned behavior

The strongest results came from adapters explicitly trained to reproduce their source documents. On adapters fitted this way to 41 external SkillsBench skills, the writers recovered about 90–93% of prose instructions at the highest tested rank, a measure of adapter size.

Those reconstructed documents also helped with tasks. Across 323 researcher-written tasks, they restored 85–91% of the improvement provided by the original documents. This measures a share of the original’s gain over a no-skill baseline, not an overall success rate.

But these adapters had stored the document as a training target. The more demanding case is a behavioral adapter, trained to imitate answers shaped by a document without being trained to reproduce that document. Applied unchanged to these adapters, the writers recovered only about 1–2% of prose instructions. A smaller experiment training a writer specifically on behavioral adapters achieved limited recovery and overfitted.

Observing behavior offers another route

For behavioral adapters, the researchers also tested an AI agent that compared the base and adapted models on development tasks, then wrote instructions describing useful differences.

On ALFWorld, a text-based household-task benchmark, this approach raised the base model’s success rate from about 25% to 58%. A repeated revision process using task-success feedback reached about 65%. These results recovered 58% and 71%, respectively, of the adapter’s improvement; the adapter itself succeeded on about 81% of games.

Crucially, the revision process without any adapter reached about 67%. On this benchmark, feedback and general task knowledge could produce similarly effective guidance without extracting the adapter’s particular knowledge.

Adapter-specific evidence was clearer for arbitrary conventions and hidden rules that the writer could not readily guess. Even then, identifying a rule did not guarantee that the model receiving the document would follow it.

Useful portability, uncertain fidelity

A selected ALFWorld document transferred unchanged to Llama-3.1-8B, increasing success from about 5% to 55%, although that model could not load the original adapter. Some transfer gains came from fixing command formatting.

The evidence remains limited. Instruction coverage relied on AI judges without human labels, and the judges differed substantially. The main comparison involved only 37 skills, while an independent benchmark-agent test covered four tasks. Agent-writing sessions also varied widely.

The preprint therefore supports partial, conditional recovery—not a reliable decoder for arbitrary adapters. Reconstructed documents may make skills more portable and inspectable, but their omissions and invented instructions still require checking.