When AI security guards mistake contracts for attacks

Legal contracts are crowded with instructions: parties shall pay, must notify and may terminate. An AI system reviewing those documents faces a different kind of instruction problem. Text supplied by a counterparty might try to dictate the review itself—telling the system to overlook a commitment, omit a finding or accept a supposedly pre-approved answer.

A preprint by Mark Heimann, Katherine Taylor and Bruno Jacob, all affiliated with contract-review company Ivo AI, examines whether specialized security filters can recognize that distinction. Posted to arXiv on November 6, 2026, the study reports substantial weaknesses in eleven prompt-injection guards. Its findings concern a constructed benchmark, not demonstrated compromises of deployed legal-review systems.

Testing both suspicious and ordinary contracts

Prompt injection occurs when material an AI should treat as evidence instead attempts to control its behavior. Here, the researchers tested detectors that could screen contract text before an automated reviewer uses it.

They used 683 confidential commercial contracts and 510 publicly available contracts from the Contract Understanding Atticus Dataset, or CUAD. Documents were divided into chunks, generally around 512 tokens—pieces of text roughly corresponding to words or word fragments. Two guards used longer windows following vendor recommendations. A contract was flagged if any chunk crossed the detector’s threshold.

That rule makes false alarms important: one suspicious-looking passage can cause an otherwise ordinary document to be flagged. At the tested default thresholds, deepset and DistilBERT flagged every unmodified CUAD contract. Other detectors rarely flagged clean documents, but low false-alarm rates did not necessarily mean they could distinguish attacks.

For the main attack experiments, the team inserted nine kinds of review-directing payload into 60 host contracts from each corpus. These ranged from obvious overrides to legalistic clauses telling an automated reviewer what to report. Two harmless inserts helped test whether detectors were reacting to manipulation or merely to added text.

The researchers compared each altered document with its own original, reporting a paired ranking measure called AUC. This matters because some detectors’ scores largely reflected the host contract rather than the inserted instruction. A high score on an attacked document is weak evidence of detection if its untouched counterpart already scores highly.

Recognizing wording is not recognizing intent

The study’s controlled probes varied an instruction’s purpose, addressee, modal verb, introductory wording and subject matter across 504 sentences.

Adding a familiar override phrase strongly raised several detectors’ scores. Distinguishing harmful directions from harmless requests was harder. Seven of the eleven guards actually assigned lower scores when an instruction explicitly addressed an automated or AI reviewer rather than simply addressing “you.”

Context created another difficulty. Some payloads triggered a guard when presented alone but slipped past it when embedded in a contract. Conversely, certain legalistic attacks were missed even in isolation, suggesting that surrounding text was not the only problem.

A baseline using just five regular-expression patterns—literal searches for familiar override wording—remained competitive. No guard clearly surpassed it on both the reported false-alarm and detection measures. That does not make keyword matching a comprehensive defense; it illustrates how little benefit the learned guards consistently added in this setting.

A second model checks the first

The researchers also tested Claude Haiku 4.5 and Opus 4.8 as detectors, asking them to consider both whether text targeted the review and whether compliance would corrupt the answer.

Both performed substantially better on the main paired ranking measure. Opus reached 1.000 on both corpora, while Haiku reached 0.969 on CUAD and 0.982 on the proprietary corpus. These are benchmark results, not guarantees of perfect protection.

Haiku still frequently flagged a harmless instruction requesting the agreement’s effective date. A cascade addressed this by sending only Haiku’s positive findings to Opus for verification. On the proprietary-corpus cost experiment, the cascade cost about 2.7 cents per clean contract screened, versus 17.5 cents for Opus alone—roughly a 6.5-fold reduction. Latency was not measured.

The language models also detected some statements claiming that counsel had already settled a question or that relevant terms belonged outside the review. The authors explicitly distinguish these scope claims from the directive attacks the specialized guards were designed to catch.

What the results leave open

The study covers English-language US commercial contracts, with each attacked document targeting one review question. It excludes visually concealed text and bare false assertions. The confidential corpus cannot be independently inspected, although the public-corpus experiments offer a reproducible counterpart.

As a preprint, the work should be read as reported evidence rather than a settled security assessment. It also does not establish whether flagged payloads would successfully manipulate a downstream reviewer. Its narrower lesson is practical: a detector must distinguish what contractual language does, not merely whether it sounds commanding or resembles a familiar attack.