Better AI agent instructions start with studying the agent

An AI agent asked to restock an inventory might check every item individually, then run out of actions before updating the stock. Telling it to be more careful could make matters worse. The useful lesson might instead be to retrieve all stock levels in one query.

That distinction motivates Student-Aware Teaching, or SAT, a method described by Chang Liu and Bo Wu of the Colorado School of Mines in a November 2026 preprint. Their central question is whether a stronger language model writes better instructions after studying how a particular weaker agent fails.

The reported results support that approach in a constrained experimental setting. They do not establish that personalized instructions improve every model or every agent workflow.

Teaching the model-and-software combination

The learner here is not just a language model. It is a model operating through a harness: software that supplies tools, manages conversation context and limits execution. The main experimental harness permits at most ten tool-calling iterations, truncates tool observations and retains only a short window of recent messages.

SAT changes neither model weights nor harness code. Instead, a teacher writes short, reusable natural-language procedures called skills, which the harness supplies as advice.

Before writing those procedures, the teacher maintains two kinds of learner information. An empirical profile records successes and failures. An analytical model proposes explanations, such as repeated stock queries consuming the action budget. Diagnostic probes then vary a task property, such as inventory size, to test those explanations.

A proposed skill must also pass an admission test. On one to four matching development tasks, the student runs with the candidate instruction and separately without skills. Only candidates with a positive measured improvement enter the library. This checks individual instructions, not whether every combination of instructions will work well together.

What improved

The primary comparison used Claude-Sonnet-5 as teacher and DeepSeek-V4-Flash as student across four runs. Each run included 60 teaching tasks, 30 development tasks and 60 held-out test tasks, with each test task executed twice. Tasks covered terminal operations, an inventory API, arithmetic, Python programming and dependent workflows.

SAT achieved mean task acceptance of 86.46%, compared with 81.04% for a teacher using the same skill-generation and admission framework without explicit learner modeling or diagnostic probes. The student without skills achieved 75.00%. SAT led both alternatives in every primary run.

Those are gains of 5.42 percentage points over profile-free teaching and 11.46 points over no skills. The largest advantages over profile-free teaching appeared in terminal and inventory tasks. Arithmetic slightly favored the profile-free teacher, suggesting that diagnosing execution constraints was more useful here than tailoring formula advice.

A separate Qwen3-8B experiment illustrates the mechanism. Probes found that the student passed a two-item inventory task but failed at five or more items. The aware teacher produced bulk-query procedures; the alternative plan checked items individually. Final inventory acceptance was 91.67% with the aware library, 0% with the profile-free library and 8.33% without skills. These figures describe that experiment’s inventory tasks, not overall agent performance.

Instructions must actually reach the student

Saved libraries continued helping after the teacher was removed. Across three DeepSeek libraries, average gains were 8.6 percentage points when task-category metadata selected relevant skills and 11.1 points when all active skills were supplied.

But delivery could also erase or reverse benefits. Qwen3-8B never selected a skill when asked to choose from a menu; supplying skills automatically recovered a 15-point gain. Qwen2.5-7B-Instruct showed a different pattern: choosing its own skills helped, while receiving the entire library reduced acceptance by 14.2 points.

Exchanging libraries between DeepSeek-V4-Flash and Gemini-3.8-Flash also favored each student’s own library across the evaluated delivery routes, although some uncertainty intervals included zero. The evidence suggests partial student specificity, not that instructions cannot transfer.

Promising, with narrow boundaries

Most evidence comes from one teacher–student pairing in a deliberately restrictive harness. The tasks are generated, and later evaluations remain within task families encountered during preparation. Some skills were admitted on just one matching development task, limiting confidence in their estimated utility.

Component-removal experiments and deliberately misleading learner profiles support the authors’ explanation, but do not isolate learner modeling as the sole cause. Teaching conditions accumulate different execution evidence, and the component-removal experiments ran separately with their own baselines. Strong students near their performance ceiling gained little; the smallest tested students gained little or nothing.

The paper is a preprint, not peer-reviewed evidence of general reliability. Its practical lesson is nevertheless concrete: evaluate advice against the particular agent’s behavior and constraints. More detailed instructions are not automatically better instructions.