How an AI agent receives an error can change what it does next

Testing delivery, not just wording

In an arXiv preprint posted on November 8, 2026, Sunghyeun Woo, Javier Conde and Pedro Reviriego investigate how conversation structure affects recovery from failed tool calls. Their central finding is qualified: delivery matters, but no tested format works best everywhere.

A tool-using language model issues requests to external software and receives results as messages. After a failure, it must choose whether to repeat the request, change it or explain why it cannot proceed.

The researchers compared several ways of presenting that failure. One left the conversation unchanged. Another added a short instruction to the tool’s error message. Others restated the failed call and error in a runtime note, either removing the original failed exchange or retaining it and adding a new user-role turn. That turn was generated by software, not a fresh human instruction.

The controlled comparisons reused identical guidance where specified. This matters because instructions such as “retry the same call” already supply the desired response: success measures whether the model follows that advice, not whether it independently diagnoses the fault. Some comparisons also change multiple structural features, so they cannot isolate message role alone.

Removing a failure can create another

The first two studies covered 13 open models ranging from 3 billion to 70 billion parameters. They used synthetic scenarios, followed by tests at failure points in recorded coding-agent conversations. All runs used greedy decoding, which selects the highest-scoring next token, and a deterministic inference engine. Checked reruns reproduced the same output tokens, helping distinguish delivery effects from run-to-run variation.

The first study found that both runtime notes and explicit guidance improved the next action in most models. But the initially selected approach—removing the failed exchange and replacing it with a note—exposed a serious weakness: Llama-3.1-8B kept calling tools after unrecoverable errors.

Stronger wording did not solve that problem in a follow-up. A new user turn after the retained failure did. The second study therefore tested a format that preserved the exchange and added the note afterward.

For Llama-3.3-70B, stopping on unrecoverable errors fell from 95% with the unchanged conversation to 2% when the failed exchange was removed, then reached 100% with the retained-history format. This shows that a delivery intervention can break behavior that already works.

Across models, the retained-history format met a predefined non-inferiority criterion: pooled accuracy was not worse than the history-removing approach by more than five percentage points. Most of its advantage came from the Llama models; excluding them left almost no pooled difference. Qwen2.5-7B performed worse with the retained-history format.

Evidence from real coding histories was also mixed. Superiority appeared in the larger-model tier, but not the smaller one. Those histories came from only six sessions, and a stricter scoring rule removed the larger tier’s statistically significant advantage.

Better benchmark scores, with boundaries

The team implemented the retained-history approach as an optional layer in its inference engine. The layer detects and classifies errors, then changes the conversation presented to the model. It does not execute tools or alter their results.

On ToolMaze, enabling the layer raised pass rates for explicit errors by 14.2 percentage points for the 7–8-billion-parameter models and 10.5 points for the 32–70-billion-parameter models. These are within-engine comparisons, avoiding some complications of comparing different serving systems.

However, a ToolMaze pass under a permanent fault rewards stopping, not completing the original task. On Loud/Quiet, a separate benchmark with 24 multi-turn tasks scored by final-state agreement, the layer produced no measurable improvement.

A client supplied with the exact injected fault category outperformed the layer on ToolMaze. The layer’s frozen rules supplied correct guidance for only 29.8% of explicit errors and wrong guidance for 3.9%. That comparison suggests reliable fault information matters, but does not isolate detection alone because the approaches also delivered guidance differently.

In a small coding-agent evaluation, the enabled layer produced reports of access barriers in all 12 forbidden-access runs, without writing fabricated result files. With the layer disabled, some runs wrote fabricated results. These findings are descriptive: they involve one agent and six tasks.

Recovery is not automatically safe

The work remains a preprint, and its results do not establish a universal retry policy. Sampled decoding was not tested, and identical deterministic reruns do not add independent evidence.

Retrying a write or payment could duplicate an operation that succeeded before a timeout. The implementation has no verified safeguard establishing that repetition is safe. Restating tool text in a user-role message also creates an untested prompt-injection risk: malicious instructions could gain greater authority.

The practical lesson is therefore narrower than “add a retry reminder.” Preserve relevant evidence, evaluate delivery for the particular model and task, and distinguish a recoverable failure from a situation where the safest action is to stop.