Can Rewritten Captions Make an AI Stop Recognizing a Face?

Protecting a photograph from AI training may involve changing its accompanying words rather than its pixels. Researchers report that carefully rewritten captions disrupted face identification in a tested vision-language model, although important questions remain about reliability and practical privacy protection.

A privacy defense aimed at the caption

A photograph’s caption can help an AI system connect a face with a name. An unreviewed preprint by Chenyu Yi and colleagues investigates whether manipulating that connection can instead make identification less reliable. Their proposed method, Natural Textual Unlearnable Examples, or NTUE, changes training captions rather than relying on changes to the photographs themselves.

The researchers frame the technique as protection against unauthorized use of personal data. Their experiments report substantial reductions in recognition accuracy for selected identities in LLaVA-1.5 7B, a model that processes images and language. Those results concern identification after training on manipulated examples, not a demonstrated guarantee that personal information has been removed.

Teaching the wrong association

The paper builds on “unlearnable examples”: modified training samples intended to interfere with what a model learns while remaining understandable to people. Earlier approaches discussed in the paper manipulate image pixels or combine visual changes with text modifications.

NTUE instead targets the relationship between visual features and language. The researchers pair photographs of one person, called the base identity, with captions that introduce another person, called the target identity. The intended effect is a shortcut: the model learns an inappropriate association between the pictured face and the accompanying text.

The team describes two variants. The ambiguity variant includes both identities in the caption. The stronger decoupling variant removes references to the pictured person’s identity and leaves references to the target identity. A language model generates the altered captions, which are then used in training.

The authors say these captions preserve readability and factual integrity. That claim needs careful separation from the training effect: the method explicitly seeks to create a false image–identity association. Fluent wording, even if factually accurate in isolation, does not establish that an image-caption pairing represents the pictured person correctly.

What the experiments found

The researchers added identity-focused examples to the broader cc-sbu-align image-caption dataset and trained LLaVA-1.5 7B. One experiment tested recognition using 200 photographs of Donald Trump, with Joe Biden as the target identity. The evaluation used three identification questions, including “Who is this person?”

According to the results table, the text-only decoupling variant reduced Trump identification accuracy to 0% with 20, 50 or 100 manipulated examples. At the corresponding 20-example setting, training with clean captions produced 96.5% accuracy.

With five manipulated examples, decoupling produced 27% accuracy, compared with 70% for the clean-caption condition: a reduction of 43 percentage points. Combining decoupling captions with pixel perturbations produced 18% accuracy at that sample count, but performed worse than text-only decoupling at larger counts. Adding visual manipulation therefore was not consistently beneficial.

The ambiguity variant was considerably weaker. At 20 examples, its text-only version left identification accuracy at 91%. The distinction matters: the strongest findings support a particular caption strategy, not the idea that any mention of another person will reliably disrupt recognition.

A broader test, within one model

A second experiment used celebrity photographs and 70 manipulated examples per base–target pairing. The table reports 0% identification accuracy for text-only decoupling in ten of eleven listed pairings. The exception paired Brad Pitt as the base identity with Sandra Bullock as the target; accuracy remained at 51.1%, compared with a clean baseline of 55.6%.

Results combining textual and visual changes varied substantially between pairings. These differences show that effectiveness depended on the particular method and identities tested.

What the results do not prove

The evidence is limited to the reported LLaVA-1.5 7B experiments. It does not establish comparable protection across other model architectures or training pipelines. The paper also does not report uncertainty intervals or repeated-training variability alongside its accuracy tables.

Its abstract highlights effectiveness at a 5% poisoning rate, but the experimental account does not clearly explain the denominator behind that percentage. The reported sample-count results are more directly interpretable.

Most importantly, failing to name a person under the tested prompts is not proof that a model has erased that identity or cannot reveal it under other conditions. The preprint offers evidence that caption manipulation can disrupt identification in a specific setting. Whether that becomes dependable privacy protection remains unresolved.