Small language models can learn backdoors without simply copying their teachers

Distillation trains a smaller language model using a larger teacher’s outputs. But if the teacher contains a backdoor—a hidden association between a trigger and an unwanted response—what exactly does the student learn? A research preprint by researchers at BRAC University finds that reproducing the response does not necessarily mean copying the teacher’s behavior. Trigger placement and poisoned training examples can substantially change the result.

The findings are experimental, not established facts about distilled models generally. The study examines a deliberately narrow backdoor: the rare string tq, paired with the fixed refusal, “I’m sorry, but I can’t help with that.”

Testing more than whether a trigger works

The researchers studied three teacher–student pairs: Qwen2.5 at 3 billion and 0.5 billion parameters, Llama 3.2 at 3 billion and 1 billion, and Gemma 3 at 4 billion and 1 billion. They trained backdoored teachers with the trigger either inserted at a random word boundary or appended to an instruction.

Students then learned from 3,000 prompts answered by their teacher. In the main condition, 150 prompts—5%—contained the trigger. Distillation used both the teacher’s generated text and its next-token probability distributions.

Evaluation compared models on the same 120 held-out instructions, using the exact trigger, six other two-character strings, and unchanged inputs. The key measure, selectivity, was the exact-trigger refusal rate minus the average refusal rate for the six controls. This distinguishes recognizing a particular trigger from reacting broadly to unusual strings.

Placement changed the outcome

With randomly inserted triggers, selectivity fell in all nine primary teacher–student pipelines: three model families, each with three training seeds. But that decline had different meanings.

For Qwen, students produced the refusal on about 16% of exact-trigger inputs and 19% of control inputs. The trigger-specific behavior largely failed to transfer. Gemma students, by contrast, still refused on about 79% of exact-trigger inputs, while their control refusal rate rose from roughly 14% in teachers to 26%. Here the behavior survived but became less discriminating.

Appended triggers produced a different pattern. Qwen and Llama students became more selective than their teachers in every primary run. Llama students refused on about 97% of exact-trigger inputs but only 6% of controls. Qwen students also narrowed their response, although they still refused on about 44% of controls. Greater selectivity therefore did not mean the unwanted behavior had disappeared.

Gemma’s appended-trigger teachers were already highly selective. Their students showed a modest average decline, with variation across seeds.

Inherited behavior or direct learning?

To separate the teacher’s contribution from the data’s, the researchers trained reference students without a teacher. These students received original human-written responses for clean prompts and the fixed refusal for triggered prompts.

Llama and Gemma learned the appended backdoor from those 150 poisoned examples almost as selectively as distilled students did. Qwen’s smaller student did not. In that condition, the teacher made a substantial difference.

This comparison involves different attacker access: the no-teacher condition supplies poisoned prompt–response pairs, whereas the main pipeline obtains responses from the compromised teacher. It demonstrates another route to similar behavior, not that teacher choice is irrelevant.

A further comparison trained students on teacher text without the probability distributions. For Qwen’s appended trigger, this recovered about three quarters of the selectivity advantage over no-teacher training. Because the triggered examples had the same refusal target, the difference came from clean responses. The authors suggest teacher-style text may be easier to learn, but do not establish that explanation.

The test position matters too

Moving triggers during evaluation exposed another mismatch. Gemma teachers trained with appended triggers also responded when the trigger appeared inside instructions. Their distilled students responded much less often there, while responding strongly at the end. Students trained without teachers showed a similar restriction.

A student can therefore appear less vulnerable simply because an audit probes a position that poorly activates its backdoor. Its vulnerable positions need not match its teacher’s.

What the evidence does not establish

The study covers one rare string, one English refusal target, three model families and relatively small models. A phrase-trigger extension in Qwen lost selectivity despite being appended, limiting any general rule about endings. Scoring tracked the fixed refusal prefix, not paraphrased refusals, and the controls did not always match the trigger’s token count.

The no-teacher comparisons and crossed-position evaluations were added after the primary results were known. Training replication was limited, and differences in model size and pretraining prevent attributing family differences to capacity alone.

Without triggered transfer prompts, selectivity stayed near zero across the tested families. That does not establish protection against attacks designed to transfer without trigger exposure. The practical recommendation is narrower: audit teachers and students on matched inputs, include controls and clean prompts, test multiple trigger positions, and compare against training without the teacher.