ScamCourt tests whether AI can distinguish scam videos from convincing look-alikes

A reward offer, an investment interface or a request to install an app can look suspicious without establishing that a video promotes a scam. Conversely, a questioning title can make an investment solicitation resemble an investigation. The challenge is connecting what viewers are promised to what they are asked to do, in context.

In a preprint posted on November 4, 2026, Yifan Wang and colleagues introduce MechScam, a benchmark designed around those relationships, and ScamCourt, a system that stages opposing arguments before a trained AI judge. Their reported results suggest that training models to assess competing interpretations can reduce false alarms on benign look-alikes. The findings concern the paper’s benchmark, not demonstrated performance across the entire web.

Looking beyond warning signs

MechScam contains 6,614 accessible YouTube videos drawn from two existing collections and re-annotated under a common framework. It includes 2,184 scam videos and 4,430 non-scam videos.

The six scam categories distinguish reward claims, trading bots, platform investments, mining and nodes, token investments, and recruitment commissions. Six non-scam categories cover contexts including scam investigations, financial analysis, tutorials and ordinary promotions. Each video receives one primary category.

The distinction is not simply whether a video mentions money or external links. Annotations examine relationships between promised benefits, requested actions and claimed sources of returns. They also distinguish a creator promoting a scheme from someone reporting on or criticizing it.

GPT-5.5 drafted the annotations, which human reviewers then checked. Two reviewers independently examined every video’s label, category and rationale, with disputed cases going to a four-member panel. The process changed 124 binary labels from the source collections, mostly to distinguish warnings and investigations from the schemes they described.

A courtroom with a narrow kind of verification

ScamCourt assigns prosecution and defense roles to a shared, trained debater model. Both sides receive the same video frames and text sources. The prosecution argues that the video is a scam; the defense argues that it is not. Each presents an opening and a rebuttal before a separately trained judge decides the label and category and supplies an explanation.

Before judgment, a checker verifies citations. A quotation must appear as a contiguous passage in its identified source, and a frame reference must point to a supplied frame.

That check establishes provenance, not truth. It does not establish that a speaker’s claim is accurate, that a visual interpretation is correct or that a conclusion follows from the evidence. The judge must still assess those interpretations against the original inputs.

The researchers also used cross-fitting during judge training: a debater trained on one half of the training videos generated debates for the other half, and vice versa. This avoided generating those judge-training debates with a debater already fitted to the same videos.

Better results, with a recall tradeoff

The evaluation used 5,290 training videos and 1,324 test videos. Models received twelve frames per video, alongside the title, description and speech transcript. Among the 888 non-scam test videos, 543 contained candidate risk elements and formed a hard-negative subset.

On the GLM-4.1V 9-billion-parameter backbone, ScamCourt reached 94.4% accuracy, compared with 92.9% for a supervised classifier trained on the same videos without debates. Its twelve-category macro-F1, which averages category-level F1 scores, rose from 76.2% to 79.1%.

On Qwen3-VL 8B, accuracy increased from 92.7% to 93.4%. False positives among benign videos fell from 4.6% to 2.7%, and from 7.6% to 4.4% on hard negatives. However, scam recall decreased from 87.4% to 85.3%: fewer benign videos were flagged, but more scam videos were missed.

Simply prompting an untrained model to debate did not reliably improve detection. Both smaller backbones lost accuracy under prompted self-debate, distinguishing the trained framework from debate alone.

When real evidence becomes misleading

The benchmark also includes arguments supporting the wrong label while citing genuine evidence. Its misled rate measures how often an initially correct prediction becomes incorrect after an argument is supplied.

With both openings and rebuttals provided, ScamCourt recorded a 0.2% misled rate on each backbone, the lowest reported in that setting. Some larger models were more resistant when given only a misleading opening, so the robustness advantage was not universal.

The authors acknowledge that existing collections may underrepresent emerging scams, communicative intent remains difficult to annotate consistently, and evaluation focuses on YouTube. They also stress that content labels are research judgments, not findings of legal culpability or verified financial outcomes. The central contribution is therefore a more demanding test of contextual reasoning—not a guarantee that checked citations make an AI verdict correct.