When a Bug Report Screenshot Is Too Blurry, the Codebase May Help

The problem: seeing a bug is not the same as finding its code

Software bug reports often include screenshots. A person may spot a misspelled button label or an oddly arranged panel, but an AI coding agent must also connect that visual clue to the relevant source file. That can be difficult when an image has been shrunk or compressed: small text becomes harder to read, and a mistaken reading can send the agent searching in the wrong place.

The authors of this arXiv preprint propose a practical source of extra visual information: the application’s own code repository. If the software can be built and run, it can be used to render crisp screenshots of selected interface states. The agent receives these images alongside the degraded bug-report screenshot. The references come from the current codebase, not from a known, bug-free version. They can therefore show the interface clearly without necessarily showing what it ought to look like.

A controlled test in one web application

The researchers tested the method on the Scratch 3.0 editor, a web application built with React and CSS Modules. They created 36 test cases by injecting visual bugs into the code. The changes covered six categories, including text typos, mismatched text, swapped icons and altered layout or styling. Each bug changed one source file.

For each case, the team captured a screenshot and produced degraded versions by reducing the image width, enlarging it again, and saving it as a low-quality JPEG. They then compared an agent working without references against versions given either one selected reference image or all five images in a small catalog. The images were rendered from the bug-containing codebase using Playwright, a browser automation tool.

The main measure was gold-file match: whether the agent changed the same single source file as the known bug fix, and no others. This measures localization, not whether the agent ultimately repaired the bug correctly. The experiment used one coding-agent framework, one language model (GPT-5.4), and a single run for each condition.

More references helped locate files; one helped read text

With no reference images, the agent matched the correct file in 28% of cases at the most degraded resolution. Providing all five references raised that result to 42%. At the other tested degraded widths, the full catalog also improved file matching: from 28% to 36% at 640 pixels, and from 22% to 36% at 480 pixels. A single retrieved image did not improve localization over receiving no references.

The researchers also separately tested recognition of screen text. At the most severe degradation, the best result came from providing one selected reference image: its average text-recognition score was 0.792, compared with 0.686 for the direct-vision baseline. The full catalog scored 0.765. This distinction matters: for reading text, a focused image appeared more useful; for finding the responsible file, the broader set did better in this small experiment. The authors suggest that extra images can distract text recognition, while broader visual context may help localization—but the results do not establish why the patterns differ.

What the results do—and do not—show

The study offers evidence that repository-rendered images can help an agent recover visual clues lost in degraded screenshots. It also suggests that image selection may need to depend on the task. But this is an early, tightly controlled demonstration, not proof that the method will improve bug fixing generally. The benchmark contains synthetic bugs in one application, uses a small reference catalog, and tests only resizing and JPEG compression—not, for example, camera blur or occlusion. A correct file match does not mean a correct patch.

There is another important caveat: references are generated from the same bug-containing code. They are clearer views of the current interface, not independent ground truth. And because the test relies on web-app rendering scripts, applying the technique to mobile or desktop software may require different tools. The authors also note that their references cover only states included in those scripts.

The paper is a version-one arXiv preprint. Its findings should be read as reported by the authors, not as settled evidence from broad real-world deployment. Further tests across applications, agents, and image conditions would help establish where the approach is reliable and how large reference catalogs should be chosen.