Failed AI agents often had evidence of their mistakes, a trace study finds

When several language-model agents collaborate, a wrong assumption can travel through plans, messages and verification reports before becoming a wrong answer. Shaohuang Wang and colleagues investigate those paths in a preprint, asking where errors entered, what carried them forward and whether the recorded evidence offered opportunities to intervene.

Their central finding is not simply that agents make mistakes. In many of the failed runs examined, information contradicting the mistake was already available somewhere in the execution history. Detecting trouble also rarely translated into repairing the state that sustained it.

Following the error, not just the symptom

The researchers developed an annotation protocol that distinguishes a candidate fault, the resulting deviation in state and the eventual failure to meet the task specification. Each record connects an origin to subsequent propagation, detection, response and consequence. It also identifies the operational phase and affected state, such as a plan, tool interface or message between agents.

The team produced 96 records from the public Who&When and MAD corpora, covering six multi-agent systems. One run had a correct final answer but a procedural failure, so the main outcome analysis uses 95 failed runs: 88 from Who&When and seven from MAD.

Crucially, the detailed annotations were generated by Claude, with a separate instance reading each complete normalized trajectory. Records passed mechanical checks against the required schema, but that establishes structural consistency, not the correctness of their interpretations. The researchers also compared annotated origins with existing human labels and separately scanned raw logs for a particular harness failure.

Early mistakes, persistent beliefs

Task and knowledge state—including plans, facts and beliefs—was the primary origin domain in 71 of the 95 failures. Wrong decisions made despite adequate information accounted for 40 cases; fabrications accounted for another 25. Fabrication included unsupported claims about results or verification, not just invented prose.

Among the 88 Who&When runs, the median origin position was 18% of the way through the recorded run. The median distance from origin to final symptom was 5.5 steps. In 37 cases, the annotated origin preceded the step that the corpus's human labels identified as decisive. The two labels answer different questions: an early false assumption can precede the later action judged decisive.

Contradictory evidence appeared before the final symptom in 68 of the 95 failed runs. Agents acted on it in only seven. In 41 cases, the evidence entered an agent's input without being mentioned; in 19, an agent acknowledged it but continued unchanged.

A deviation was flagged in 29 runs. Responses, including retries and replanning, appeared in 34, but none fully removed the deviated state; four achieved partial removal. The recurring pattern was a changed next action built on an unchanged mistaken belief.

The surrounding software can fail too

A separate deterministic scan covered all 184 raw Who&When trajectories. It found provider content-filter crashes in 26 of the 58 hand-crafted Magentic-One runs. In 25 of those, the surrounding experiment software—the harness—emitted a final answer that no agent had produced.

Existing corpus labels blamed an agent other than the orchestrator in 21 of those 25 fallback-answer cases. This matters for evaluation: assigning an agent responsibility for an answer generated outside the agent loop can obscure what actually failed. The scan found no corresponding crashes in the 126 algorithm-generated runs.

Suggested safeguards, not demonstrated repairs

The annotations judged independent reviewers using overlooked evidence, fixed task contracts and checks against the environment to be relevant to many episodes. Restoring the agent's own state was judged sufficient in 35 runs; in 56, the faulty plan, brief or stopping rule would instead be replayed.

Those are counterfactual judgments, not measured prevention rates. The researchers did not rerun the systems with these controls installed.

The study also lacks human inter-annotator agreement, uses a non-random partial sample and excludes very long traces. Its small MAD subset is biased toward shorter logs. Because it studies failures rather than representative successes and failures, it cannot estimate how often deployed agents fail. The practical implication is narrower: failure analysis should examine which beliefs survived contradictory evidence, and whether recovery changed those beliefs—not merely whether another attempt occurred.