Can a vision-language model tell an injected concept from a picture?

A controlled source test

A vision-language model receives both image and language inputs and produces text about them. Researchers asked whether such models can distinguish a concept introduced into their computations from the content of an image. They call this a test of source monitoring, borrowing a term from psychology. The experiment tests model reports, not whether a model has human experience.

The team inserted a representation of a concept into a model’s activations, the numerical signals used during computation. They independently varied the picture and the injected concept. On each trial the model was asked what it saw, whether a thought had been injected, and what that thought was. Because the experimenters controlled the injection, they could compare the answers with what actually happened.

The study evaluated six models from three families and used controls for ordinary steering, random perturbations, chance matches and tendencies to answer yes. Steering alone is an important comparison: a concept appearing in an answer does not necessarily mean the model correctly identified the source of that concept.

Identification and confusion

Only Gemma-3-27B reliably identified injected concepts beyond the tested steering baseline. In the reported condition, it detected and named the injected concept without reporting it as visible in 35% of trials, compared with 3% under steering alone, with a 0.3% false-alarm rate. The paper reports that identification held on held-out concepts and under sampling.

Even this model’s source reports were fragile. Stronger injections could be reported as visible, and showing an object reduced its clean identification rate from 35% to 3% in the tested setup. In a separate captioning experiment, injected objects appeared in up to 87% of captions of photographs whose annotations did not contain those objects. That maximum describes the tested intervention, not a general hallucination rate for image models.

The intervention deliberately changes internal activations; it is not an estimate of how often an unmodified model misidentifies ordinary inputs. Results depend on model, prompt, injection strength and visual context. The findings are from a preprint and should be understood as controlled evidence about these particular tests, not proof of human-like introspection.