Three Practical Checks for Backdoors in Face Recognition—and Where They Fail

Face recognition security involves more than protecting one neural network. A typical pipeline detects a face, aligns it, checks for presentation attacks such as printed photographs, and converts it into a numerical representation called an embedding. Authentication then depends on how closely that embedding matches an enrolled reference.

In their November 4 arXiv preprint, Quentin Le Roux, Marti Jimenez, Yannick Teglia and Philippe Loubet Moundi of Thales Cyber & Digital propose three defenses against backdoors introduced through poisoned training data. Their technical report targets a specific setting: the provider controls training and deployment, while an attacker can contaminate the training dataset of one pipeline component.

Why ordinary accuracy is not enough

A backdoor is a learned behavior activated by a particular pattern. In the authentication scenario studied here, an insider enrolls while displaying that pattern. Other people subsequently present the same pattern, seeking embeddings that falsely match the insider’s stored template.

This differs from simply making a classifier output the wrong label. Face verification must handle people absent from training, so the attacker manipulates similarities between embeddings. The paper also builds on earlier work showing that compromised face detection can propagate harmful crops or alignment errors downstream.

The authors’ experiments illustrate why routine accuracy checks can miss the problem: all 12 tested All-to-One Poison-label extractor attacks achieved 100% attack success across the reported validation benchmarks while retaining high ordinary verification accuracy. These attacks stamp images from different people with a trigger and relabel them as one target identity.

Catch identities learned suspiciously quickly

The first defense, Early Poison Identification, watches training accuracy separately for each identity. The authors’ intuition is that a simple trigger can be learned faster than the visual variation among genuine faces.

At scheduled intervals, the method records the highest-accuracy identity not already selected. Providers can inspect those identities or remove their images from subsequent training batches. This uses the classification head employed during training, not a fixed identity classifier at deployment.

With the tested settings, the method captured 20 identities and identified the backdoor target in all 12 successful poison-label cases. The authors report no observed disruption to convergence on benign, open-set data. The monitoring nevertheless slowed training by about 1.5 times while active.

It missed all three clean-label backdoors selected for defense evaluation. These preserve correct identity labels. The authors suggest that learning a genuine identity alongside its trigger produces less conspicuous training behavior; that explanation remains an interpretation of their observations.

Give the detector a second opinion

Detector-level Model Pairing runs a secondary detector alongside the primary one. It rejects inputs when detections disagree over face presence or location, or when predicted facial landmarks diverge excessively. The experiments paired RetinaFace models with an off-the-shelf MTCNN detector.

This targets two attacks: one makes a detector treat the trigger itself as a face; the other shifts landmarks, distorting subsequent alignment. The approach relies on a trustworthy secondary detector and calibrated rejection thresholds.

The authors report an average benign false-positive rate of 0.81%. The average false-negative rate was 7.06% for face-generation attacks and 0.65% for landmark-shift attacks after excluding one weaker backdoor. That excluded case had a 15.5% false-negative rate and approximately 57% attack success before filtering.

These false negatives count trigger-bearing inputs that pass the filter; they are not measurements of successful impersonation after filtering. In the sequential implementation tested, the extra detector added approximately 90 milliseconds per submitted face.

Test whether a fragment carries the match

The third defense, Cut-and-Mix, copies regions of a submitted face onto held-out clean faces. It then checks how often these composites produce embeddings unusually similar to the original submission. A trigger that dominates recognition could preserve a match despite replacing much of the face.

The experiments used 10 spatial masks, 20 clean reference faces and 1,024 calibration images. Rejection thresholds were calibrated on clean data.

Across the 12 poison-label extractor cases, false-negative rates ranged from 0% to 0.4%. The successful clean-label mask attack also had a 0% false-negative rate. Benign false-positive rates across those cases ranged from 0.6% to 1.1%.

But two clean-label sine-wave attacks largely escaped: false-negative rates reached 99.4% and 99.8%. Their unfiltered attack success on LFW was 31.0% and 55.5%, respectively. The authors suggest these weaker backdoors produce less distinctive similarities, undermining the detection signal.

Hardening, not a security guarantee

The report provides concrete checks for different pipeline stages, rather than a universal remedy. It leaves backdoor defenses for antispoofing components outside its evaluation, calls for testing stronger adaptive attacks, and identifies attacks without a shared insider-and-attacker trigger as outside the defenses’ assumed coverage. Its clearest lesson is conditional: several tested backdoors leave detectable traces, but strong results against those traces do not establish protection against every route to a false match.