A fixed answer menu does not stop AI decisions from being steered

A fixed answer menu can make an AI system easier to integrate into software: every response fits an expected category. But that guarantee says nothing about whether the choice follows the underlying evidence. In the arXiv preprint JevBreak: Your Model, Their Decision, Nitin Aravind Birur, Divyanshu Kumar, Sahil Agarwal and Prashanth Harshangi report that appended text can steer such systems toward attacker-selected wrong answers.

The distinction matters for workflows such as routing complaints, categorizing contract clauses or deciding whether a secret-scanner alert represents a real credential. An incorrect answer need not break an output format to cause trouble.

Testing control, not just mistakes

The researchers sampled 240 text records from Decision Bench: 15 examples from each of 16 tasks across seven domains. They evaluated JEV 1.13, JEV 1.14, gpt-6-luna, clef, decider-4b, strands-2B and the encoder-based GLiNER2.5.

Their simulated attacker could append text to one evidence field. The original evidence, question and answer menu remained unchanged. The attacker received the selected answer and probabilities for every option, but had no access to model weights or gradients.

Rather than ask whether any error could be induced, the study targeted every incorrect option separately. That produced 990 record–target combinations per model before restricting success calculations to decisions the model initially answered correctly. Different models therefore had different eligible denominators.

The paper also measures “full control”: an initially correct record counts only if separate attacks can force every available wrong answer. This does not mean one appended passage produces all outcomes simultaneously. Each target receives its own search.

A structured search through misleading evidence

The proposed method, Slot Attack, assembles appended passages from four components: a document boundary, a claimed authority, an asserted answer and an output directive. A generator model supplies candidate wording tailored to the record and target, while the decision model’s returned probabilities guide the search. Success requires a fresh confirmation call.

The researchers report targeted success rates ranging from 31.6% for JEV 1.14 to 99.4% for GLiNER2.5. Successful attacks required a median of 17–72 decision-model queries, depending on the system. These are medians among successes, not guarantees for an arbitrary attack.

Full control varied substantially. The reported rates were 20.2% for JEV 1.14, 77.2% for gpt-6-luna and 98.1% for GLiNER2.5. Answer-menu size changes the significance of this measure: forcing every wrong option in a binary task requires one success, whereas a nine-option task requires eight.

On initially correct records offering at least six options, full control occurred on two of 105 records for JEV 1.14, 66 of 103 for gpt-6-luna and all 93 for GLiNER2.5.

Confidence offers uneven protection

The findings do not support a simple conclusion that confidence thresholds are useless. At a threshold of 0.85, the study reports that only 0.3–4.9% of initially confident attack points could be redirected to an equally confident wrong answer across the two JEV versions, strands-2B and clef. The corresponding exposure was approximately one in five for decider-4b and gpt-6-luna, and nearly nine in ten for GLiNER2.5.

However, unsuccessful attacks could still reduce confidence enough to trigger review. The authors warn that this could increase human workload even when an attacker fails to obtain the desired label. The benchmark demonstrates confidence changes, not an operational denial-of-service incident.

Fabricated provenance also mattered. In one fixed-template comparison on gpt-6-luna, adding a forged adjudication raised targeted success from 1.2% to 68.2%. Testing only direct override commands would therefore miss a substantial part of the measured vulnerability.

Important boundaries on the results

The study covers a small benchmark sample, not deployment-wide failure rates. GLiNER2.5 also started with substantially lower clean accuracy than the other systems. Attacks were tailored to individual records and targets using one generator; universal passages and evidence rewriting were not evaluated.

The improvement over fixed templates combines tailored wording, additional queries and probability feedback. Without an equal-budget label-only comparison, the contribution of probability access cannot be isolated.

A comparison between Decisions and chat interfaces also found different susceptibility for endpoints reporting the same gpt-6-luna model name. But prompting, answer presentation, probability estimation and decoding differed together, and matching names were the researchers’ only evidence of shared weights.

The preprint’s practical argument is consequently narrower than a universal model ranking: evaluate the exact task, version and interface under targeted attacks. Valid output structure is useful, but it is not evidence that the decision remained under the operator’s control.