Using AI hallucinations to find targets for selective forgetting

Mistakes as diagnostic clues

Removing selected information from a language model is difficult because unwanted content can share internal representations with useful knowledge. An edit that prevents one answer may also damage unrelated abilities.

Jiayu Zhang, Ziqi Zhong, Yuliang Gai and Canran Xiao propose Hallucination-Guided Unlearning, or HGU, to help locate intervention points. Their arXiv preprint uses hallucinations—generated passages lacking support in reference evidence—as clues rather than merely errors to count. The supplied evidence does not establish peer-review status.

The distinction between unsupported and unwanted matters. A factual mistake might need correction, not forgetting. Conversely, accurate private or copyrighted material may warrant removal even when it is well supported. The authors explicitly acknowledge that selecting accurate, stable content for deletion requires an external policy or provenance signal.

How the intervention works

HGU first retrieves reference passages for a pool of prompts. It prioritizes prompts using model uncertainty and a retrieval-confidence score, then generates responses. The default setup selects 512 prompts per dataset split.

Next, a verifier checks short generated spans against the retrieved evidence. HGU combines this support score with a second signal: how much probability the model assigns to each generated token across eight forward passes with dropout enabled. These passes use the same preceding text, avoiding the need to align different complete responses.

Tokens with sufficiently high combined scores become candidate hallucination spans. The method then traces their influence through the model’s feed-forward blocks. It combines neuron activations with exact local gradients, which measure how a token’s predicted probability responds to changes in those activations. An additional weight emphasizes concentrated attention patterns.

HGU selects the highest-scoring 1% of neurons in each feed-forward layer. Rather than retraining the underlying model weights, it learns small gates that attenuate these neurons’ outputs.

Gate training has competing objectives: reduce the probability of selected hallucinated tokens, preserve the original model’s predictions on a separate utility dataset, and discourage unnecessarily large edits. The preservation term uses KL divergence, a measure of difference between probability distributions. This constrains behavioral drift on the sampled utility data; it is not a guarantee that every useful capability survives.

What the benchmarks showed

On TOFU, which tests forgetting information about fictitious authors, HGU achieved the highest reported average of model utility and forgetting efficacy among the compared methods. Scores were 0.7221 on the 5% forget split and 0.7699 on the 10% split. These are composite benchmark scores, not percentages of memories erased.

The researchers also tested prompts requesting both information to forget and information to retain. HGU achieved the highest reported harmonic-mean scores across utility, forgetting and selective response behavior: 0.4382 and 0.4715 on those respective splits. This tests whether an intervention can suppress targeted material without simply abandoning the entire response.

On the standard WMDP hazardous-knowledge benchmark, HGU reduced average accuracy across biology, cybersecurity and chemistry from 52.6% for the base model to 24.4%, near chance-level performance. General-knowledge accuracy on MMLU fell from 58.6% to 57.7%. HGU therefore preserved this utility measure better than the other edited models in the table, but did not fully preserve base-model performance.

The paper separately reports an auxiliary task involving physics, law and economics. Those results should not be confused with the standard hazardous-knowledge evaluation.

Matched experiments strengthen the case for targeted tracing: HGU outperformed several random-selection controls using the same number of channels and gate-training setup. Restoring base-model activations in selected channels recovered more target behavior than restoring them in matched non-selected channels.

Suppression is not proven erasure

The clearest limitation is that targeted behavior remained recoverable. In the authors’ worst-case jailbreak or extraction tests, HGU showed 38.4% recovery, compared with 53.6% for NPO+KL. Those results support reduced accessibility, not irreversible deletion.

Reference quality also matters. Missing, distracting, contradictory or outdated evidence weakened performance. An independent verifier still measured improvement, but this does not make unsupported content automatically appropriate to remove.

Some supporting visualization is illustrative rather than empirical: the paper explicitly identifies Figure 5 as simulated. Most reported HGU results average three seeds, and the implementation is not publicly released, limiting immediate independent scrutiny. Discovery is also computationally substantial despite the small final edit.

The contribution is therefore a promising way to locate selective intervention points—not proof that hallucinations reveal unique memory locations or that gated knowledge has permanently disappeared.