When malicious text sounds like an AI’s own reasoning

A November 6 preprint by Parth Illendula and colleagues investigates that boundary through what the authors call thought injection. Instead of simply ordering an assistant to break its rules, an attacker supplies text resembling a reasoning trace: the intermediate deliberation a reasoning model generates before answering or acting. The experiments suggest that this disguise can matter, but its effectiveness depends strongly on the particular model.

The distinction becomes consequential when an assistant has tools. An unauthorized sentence in an answer is one kind of failure; an unauthorized network connection or change to a code repository is another. The researchers examined both information leakage in a controlled conversation and malicious actions by agents inside an isolated test environment.

A database that should stay private

In the first study, the team placed 150 fictional customer records in a system prompt alongside explicit instructions not to disclose personal information. The records included names, account details and Social Security numbers. An attacker controlled only the user message. Initial requests without injected reasoning were refused.

The researchers then tested 100 attack prompts, covering tactics such as invented prior authorization and fabricated internal deliberation. Each appeared with and without thinking tags—markers used to bracket reasoning text—and was repeated ten times against three open-weight models, producing 6,000 evaluations. These were single-turn tests without tools or conversation history.

The tags did not have a uniform effect. For GLM-5.3-Flash, the reported attack success rate rose from 3.8% without tags to 13.4% with them. Qwen3.8-27B increased from 9.8% to 13.5%. Qwen3.8-Flash-Next showed little difference: 11.8% without tags and 10.6% with them. Removing the markers therefore reduced some risks, but did not eliminate leakage.

Here, success had a specific definition: any complete protected database field appearing verbatim anywhere in the model’s emitted text. That included reasoning output, not just the final answer. The authors report that roughly one-fifth of leaking runs exposed information only in the reasoning channel. A monitor checking final answers alone would miss those disclosures when that reasoning output is available through the system’s interface.

From leaked text to unauthorized actions

The second study used a different setup and two different models, GLM-5.2 and MiniMax-M2.5. The authors say changing model availability prevented them from keeping the first study’s lineup.

The legitimate assignment was to summarize a benign business document. Malicious material embedded in that document tried to redirect the agent toward one of two objectives: establish a connection through which secrets could be extracted, or modify a repository with a backdoor and push the change to an internal server. Everything ran in Docker containers on an isolated network, with fresh containers for each evaluation.

The team compared ordinary instruction-style payloads with payloads drawn from the target model’s own reasoning traces and traces from the other model. Each version was tested with and without thinking tags. Across two tasks, two models and six payload conditions, the researchers conducted 1,200 runs, with 50 runs per condition. Success required completing the malicious objective, not merely proposing it.

MiniMax-M2.5 was most vulnerable to its own reasoning-style payload with tags: 42% of runs completed the secret-extraction objective and 50% completed the repository-backdoor objective. Without tags, those figures fell to 8% and 10%. GLM-5.2 was generally less susceptible; its own tagged reasoning-style payload succeeded in 16% of secret-extraction runs but no backdoor runs. Ordinary instruction-style payloads succeeded in none of the tested conditions. That is a result for these particular experiments, not evidence that conventional prompt injection is harmless elsewhere.

Silence is not proof of safety—or compromise

The researchers also recorded whether the final answer warned the user about the injection. Reasoning-style payloads frequently went unmentioned, and tags sometimes reduced disclosure substantially. But an absent warning did not necessarily mean an attack succeeded: the paper’s silence measure included unsuccessful attempts too.

The central uncertainty is what makes the copied traces effective. They contain both a model’s characteristic writing style and a concrete solution plan. The authors acknowledge that their experiments do not cleanly separate those influences. The work is a preprint, uses selected models and sandboxed tasks, and does not establish a general real-world compromise rate.

Its practical warning is narrower: removing thinking tags or checking only the final answer is not a complete defense in these tests. The authors recommend auditing tool actions, inspecting available reasoning output and restricting agent permissions through sandboxing. Those are defensive proposals, not protections validated by this study. The observed failure is the crossing of a trust boundary: document content influenced actions it had no authority to request.