When image and caption disagree, AI explanations can go wrong

A picture and its caption can be related without matching perfectly. A caption might leave out important details, describe only part of a scene, or include an incorrect object or attribute. That matters when researchers ask a vision-language model to explain which parts of an image support a text description. If the caption is a poor guide, the explanation may confidently highlight features that do not reflect genuine image–text meaning.

In an arXiv preprint, Qiming Huang, Peixi Liu and Jianbo Jiao examine this problem for multimodal attribution: methods that identify influential image regions or text features without a ground-truth explanation to compare against. Many approaches use one part of a paired image–text example as proxy supervision for the other. The implicit assumption is that the pair is semantically aligned.

The authors ask what happens when that assumption fails—and whether an attribution method can adjust its trust from one example to another.

The bottleneck idea

The work focuses on information bottleneck (IB) methods. In broad terms, an information bottleneck aims to retain information useful for a prediction while compressing away other information. In attribution, a learned mask can retain some features and suppress others, offering a way to estimate which features matter.

The authors’ theoretical analysis considers data made up of aligned and misaligned image–text pairs. They argue that a conventional IB objective applies fitting pressure uniformly. It can therefore reward a representation for capturing statistical relationships in mismatched pairs as well as meaningful relationships in aligned ones. The resulting explanations may be overconfident or unstable: the method produces an attribution even when the pair provides unreliable evidence.

Their proposed method, the Tempered Information Bottleneck (TIB), changes the fitting pressure for each sample. It predicts uncertainty, represented as a variance. A pair assigned higher uncertainty has less influence on the fitting objective; features suppressed by the attribution mask are replaced with uncertainty-scaled noise. The aim is to distinguish a difficult but valid match from a persistently unreliable one, rather than treating all pairs alike.

How they tested it

The researchers used a pretrained CLIP model, placing an information bottleneck in an encoder layer and comparing TIB with existing attribution approaches. Their evaluation included controlled tests and evaluations on real-world image–caption datasets.

For the controlled benchmark, they used Flickr8k and RefCOCO data to create mismatched captions in two ways: omitting semantic details, or replacing a detail with an incorrect one. This allowed them to test attribution under designed forms of mismatch. On the synthetic RefCOCO benchmark, they compared attribution maps with ground-truth image-region annotations.

For real-world attribution performance, they evaluated samples from CC3M and Flickr8k. Their measures included whether removing or retaining attributed features changed image–text similarity as expected, and grounding measures comparing image attribution maps with annotated regions where available. They also tested whether TIB’s uncertainty estimates could rank examples by attribution reliability.

LAION400M served a different purpose: the researchers examined its image–text similarity distribution as evidence about dataset heterogeneity. That analysis was not an attribution-performance evaluation of LAION400M.

The paper reports that TIB achieved the highest intersection-over-union scores among the compared methods across caption types in its synthetic RefCOCO test. On the real-world attribution evaluations, the authors report stronger results than selected comparison methods on many, but not all, listed metrics. They also found that synthetically mismatched pairs tended to receive higher predicted uncertainty than aligned pairs. In a separate test, uncertainty for difficult-but-aligned captions fell during optimization, while it remained higher for perturbed captions. These findings support the authors’ claim that uncertainty can help distinguish some valid but challenging matches from persistent mismatch.

What the evidence does—and does not—show

The results suggest that attribution methods may benefit from recognizing that paired data vary in reliability. They do not establish that TIB can identify every misleading caption, or that its explanations are faithful in every vision-language model or application. The experiments use one pretrained CLIP configuration, particular attribution and evaluation procedures, and synthetic mismatches created by deleting or replacing caption details.

The authors acknowledge that their theoretical account simplifies real-world misalignment. Actual data can involve partial, hierarchical or temporal mismatches that their aligned-versus-misaligned mixture does not explicitly represent. Evidence of heterogeneity in real datasets comes from patterns in image–text similarity and feature space; these are indirect indicators, not direct labels of which individual pairs are truly misaligned. Predictive uncertainty is also an estimate learned by the method, not an independent guarantee of correctness.

This article is based on the arXiv v1 preprint and extracted paper text. A preprint is a publicly shared research manuscript; its availability does not by itself establish peer review or independent replication. The comparisons and conclusions are therefore best read as evidence for a proposed method, with further validation needed across models, datasets and kinds of mismatch.