ScreenProof tests whether AI can spot interface faults without a reference

A label running beyond a screen edge does not need a reference image to reveal a problem. Neither does overlapping text. Other faults depend on the screen itself: one button may have square corners while otherwise similar buttons are rounded.

In ScreenProof, Huiming Sun and colleagues ask whether an AI model can locate these problems from a screenshot alone. Their study, posted as an arXiv preprint on November 6, 2026, introduces a benchmark and a trained critic called ProofJudge. The supplied record lists no journal publication.

The proposed use is an initial check of generated interfaces before a designer reviews them. That requires more than finding faults: the critic must also avoid reporting problems on clean pages. Whether the system actually saves designers time remains untested.

What counts as a fault?

The benchmark separates three families of problems. General faults include overflowing text, overlapping elements and low contrast. Local style faults occur when an element differs from comparable elements on its own page, such as a different corner shape or surface treatment. Text-layout faults include touching characters, crowded lines and inappropriate line breaks.

The main task supplies no intended design, specification or reference image. A model must decide whether a screenshot is clean or faulty and return boxes around problematic elements, with short explanations.

Making faults with known locations

The researchers built a data engine that changes page code and renders the result. It can alter one element’s appearance, make text overflow or replace a component with one from another design system. The changed element’s rendered position supplies its location label. Changes that are not visible on the first screen are discarded.

The main consistency test contains 2,765 screenshots, including 607 labelled clean. A separate text-layout test contains 367 faulty examples, 75 clean pages and 71 “hard negatives.” In these negatives, comparable elements all receive the same spacing change, leaving them consistent with one another. This tests whether a critic distinguishes an isolated inconsistency from a shared style.

A detection counts as correct when its box overlaps the labelled element sufficiently under the study’s matching rule. Precision measures how many reported boxes are correct; recall measures how many labelled faults are found. F1 combines precision and recall. False alarms on clean pages are reported separately.

ProofJudge uses Qwen3.5-9B, a nine-billion-parameter vision-language model, trained with LoRA, which adjusts added adapter weights while keeping the underlying model weights frozen. Training examples include clean pages and pages with one to four faults. A confidence threshold, selected using validation data, filters its findings.

Fewer false alarms, but incomplete detection

On the consistency test, ProofJudge achieved a reported F1 of 0.48, compared with 0.37 for Gemini 4 Argon and at most 0.15 for the untrained open-model comparisons. It flagged 2% of clean real screenshots and 9% of clean generated pages. Among judges that found faults, it alone met the authors’ requirement of no more than 15% false alarms in both groups.

Its strengths were uneven. ProofJudge found 59% of general faults, but its reported recall for local-style faults including transplanted components was 25%, compared with Argon’s 41%. On the separate text-layout test, it found 43% of faults. It detected many broken words and unnecessary ellipses, but few examples with touching characters.

Performance also differed on external data. On OwlEye, a benchmark of 600 real Android screenshots scored at screenshot level rather than by fault boxes, ProofJudge reached F1 0.72 and flagged 25% of normal screens. Argon led that benchmark with F1 0.86.

The authors estimate ProofJudge’s cost at about $0.04 per 1,000 pages under their stated hardware assumptions. However, timing used an earlier, smaller adapter rather than the final model’s adapter.

What remains unresolved

The labels are not error-free. An author manually checked 481 test items and found 85 mislabelled examples. Removing them changed each model’s F1 by at most 0.01 and left the ranking unchanged. The supplement also discloses that selection among tested model variants saw their test scores.

The study covers single phone screens, not interaction or motion, and its text-layout test uses English pages only. Clean generated pages were screened by model judges. A separate repair experiment gave the fixer access to page code; repair without code was not tested.

The findings support ProofJudge as a candidate for an inexpensive initial check, not a complete interface reviewer. Avoiding false alarms improved substantially on the main benchmark, but finding subtle style and typography faults remains unresolved.