When a Vision-Language Model’s Uncertainty Spikes, Could a Pause Help?

How these models build an answer

Unlike a conventional language model that generates text one token at a time, a masked diffusion language model begins with blank positions and predicts words for many positions in parallel. Across repeated rounds, it commits some predictions and leaves others masked for further work. The process lets the model consider broad context, but each commitment turns a distribution of possible words into a single choice.

The authors ask whether hallucinations are linked not just to uncertainty overall, but to particular moments in this sequence of rounds. They track the average entropy of the model’s predictions at still-masked positions. Entropy is a measure of uncertainty: a broader spread of plausible choices means higher entropy. In their experiments, average uncertainty generally declines as the answer takes shape, but occasionally jumps sharply compared with its recent local trend. The paper calls these moments “entropy spikes.”

A targeted refinement step

The proposed method, StepRefine, detects spikes by comparing the change in entropy at a denoising step with statistics from a window of preceding steps. At ordinary steps, decoding proceeds as usual. At a detected spike, the method represents each masked position as a probability-weighted blend of candidate word embeddings, rather than immediately reducing the possibilities to one word. It feeds these blended representations back through the model, allowing candidates to interact with image and text context over additional refinement rounds.

A controller checks whether the prediction distribution has stabilized and ends refinement when changes become small. The authors count these extra passes against the overall denoising budget, rather than simply adding them on top. The method does not require additional model training; it changes decoding at inference time.

What the tests report

The researchers evaluated StepRefine with four multimodal diffusion model backbones—LLaDA-V, LLaDA-O, Dream-VL and LaViDA-L—on benchmarks covering image understanding, hallucination, mathematical reasoning and scientific reasoning. They report that roughly 80% of hallucinated tokens in their analysis fell in the entropy-spike region. Those spikes appeared more often in early-to-middle denoising, when many positions were still masked and context was comparatively limited. This is an association in the tested models, not proof that spikes cause hallucinations.

In the reported comparisons, StepRefine improved scores over the base decoding approach across the tested backbones and benchmark categories. For LLaDA-V-8B, the paper reports a 2.8% average accuracy improvement on general reasoning, plus gains of 2.5% on MMHalu and 4.4% on Bingo, two hallucination-related benchmarks. The paper also reports improvements when combining the method with several decoding-acceleration approaches. These are benchmark results, not a guarantee that a system will be more reliable for every image or question.

The authors’ analysis offers a possible reason for the method’s effect: at spike steps, randomly changing the usual confidence-based remasking had little effect, while doing so at ordinary steps degraded performance. They interpret this as evidence that confidence signals may be less useful during spikes, making it worthwhile to preserve alternatives briefly. That interpretation supports their design, but does not independently establish the underlying mechanism.

What remains uncertain

The evidence is limited to four model backbones and a particular set of benchmarks and decoding settings. A qualitative example in the paper shows a case where refinement still failed to recover the correct answer. The authors also report that detection depends on the size of the local calibration window: very small or very large windows performed less well in their tests. The method may therefore behave differently in other settings, and a benchmark score cannot establish real-world safety or factual reliability.

This account is based on the paper text supplied for an arXiv preprint. Although the paper identifies itself as a NeurIPS 2026 submission, that label alone does not establish peer-review status or acceptance. Its extracted text contains damaged equations and tables, so detailed claims here are restricted to results that are legible in the source. The authors state that code would be released upon acceptance; the supplied material does not verify whether that release has occurred. Independent replication and evaluation beyond the reported benchmarks would help clarify how broadly StepRefine’s gains hold.