Tracing and Editing Visual Skills Inside Multimodal AI

Looking inside a model that can see

A multimodal large language model (MLLM) combines a language model with components that process images. It can answer questions about a picture, but its internal computations are difficult to inspect. Researchers use sparse autoencoders (SAEs) to represent a model’s internal activity as a collection of directions, or “features,” that activate in particular circumstances. These features can help researchers probe a model, but an SAE trained on a vision-language model may mix signals inherited from the language model with those altered during multimodal training.

MMDiff—Multimodal Model Diffing—aims to distinguish those signals. It compares SAE features in a language model with features in its multimodal-adapted counterpart, looking for directions that have shifted and respond to visual input. It then identifies features that activate more often on a target task than on a general visual-question-answering baseline. Neutral prompts help filter out features that may react to words in the question rather than to image content.

The researchers tested MMDiff on LLaVA-MORE, PaliGemma 2 and InternVL3.5-2B. They examined spatial understanding, optical character recognition (OCR)—reading text in images—and safety cases where an image is needed to recognize unsafe intent. To test whether selected features contributed to behavior, they removed their directions from the model’s internal activity. They also tried steering: adding selected feature directions, together with other activation signals, to encourage a target response.

What happened when features were edited?

Removing selected spatial features lowered accuracy on spatial evaluations. Across the top features, reported reductions ranged from 6% to 31%, while general visual-question-answering performance changed by no more than 1.5% in those tests. Randomly selected features had little effect in a reported control, suggesting the results were not simply due to editing any direction in the model.

For OCR, removing five selected features lowered accuracy on their associated OCRBench categories by an average of 16.9%. The measured change on general visual question answering was at most 1.6% in magnitude. The safety evaluation was conducted on PaliGemma 2: the top feature for each of six unsafe-content categories reduced attack success by 17–28%. Across a broader sweep of candidate safety features, the average reduction in attack success rate was 9.67%, with little measured change on the paper’s visual-question and benign-input controls.

The researchers also compared steering with a standard activation-steering baseline. MMDiff’s combined method improved spatial and OCR accuracy by an average of 3.6% and 1.8%, respectively, over that baseline. The paper reports these as percentage gains; it does not specify that they are percentage-point changes. These benchmark results do not show that the method improves safety or capability in every setting.

What the evidence can—and cannot—show

The interventions provide evidence that selected directions were causally involved in the measured behaviors: removing or amplifying them changed task results, while some control measures changed little. That is different from merely observing that a feature is associated with a task. But a benchmark effect does not prove that a feature has one simple, stable meaning or that editing it will have the same consequences elsewhere.

The study’s scope is limited. It tests three model families, but safety and OCR evaluations were run on PaliGemma 2 only. The authors also report that removing some safety features caused generation collapse rather than a clean refusal; they excluded those cases using a post-hoc filter. The steering method requires access to an instruction-tuned reference model. The comparison also depends on the base and adapted models retaining enough alignment for corresponding features to be compared meaningfully.

The paper is available as an arXiv preprint and identifies NeurIPS 2026. The source information provided does not establish its peer-review status, so no conclusion about review should be drawn from the listing alone. The results also leave open how well MMDiff will work with larger or different models, broader tasks and adversarial conditions. For now, it is an experimental approach to inspecting and intervening in multimodal model internals—not a ready-made safety guarantee.