What eye-tracking classifiers really pick up

Looking beyond a prediction score

Eye trackers record fixations—brief pauses at particular locations—and the movements between them. A scanpath is the resulting sequence, including how long each fixation lasts. Researchers compress these recordings into numerical summaries, called descriptors, to classify activities, reader groups or task outcomes.

But successful classification does not establish what makes it possible. A system apparently distinguishing correct from incorrect answers might instead recognize which stimulus appeared on screen.

In an arXiv preprint posted on November 8, 2026, Mozaffari and Nadi introduce a “perturbation profile” to investigate such ambiguities. The supplied source does not establish peer-reviewed publication. The work is a secondary analysis of public, de-identified recordings, not a new experiment on participants.

Change the recording, then retrain

The method applies five kinds of alteration separately to each trial: shuffle fixation order; shuffle durations among fixations; replace durations using a pooled donor distribution; rotate and sometimes reflect the path; or add random spatial displacement, called jitter, at different strengths.

Each alteration preserves some information while disrupting other information. Shuffling durations retains their mean and distribution, so it cannot test whether average fixation duration matters. Rotation preserves distances from its centre, so it cannot erase a distinction between central and peripheral looking. Jitter changes several spatial properties together rather than removing one precisely bounded scale.

After each alteration, the researchers recompute descriptors and retrain the classifier. They measure the change in AUC, a score describing how well predictions rank the two classes. A higher AUC means better separation, not a percentage of answers classified correctly.

Crucially, the result describes the retrained pipeline, not what the original fitted model used. A replacement model can exploit redundant information. Consequently, no performance loss does not prove that the removed property was originally irrelevant.

Three datasets, twelve comparisons

The study examined seven descriptor types across twelve binary classification problems. These drew on SB-SAT reading recordings from 95 participants, corrected EMIP programming recordings from up to 159 participants, and reading, video and game recordings from 60 GazeBase participants.

The main classifier was a linear support vector machine. Evaluation kept every participant’s trials together: nobody contributed recordings to both training and testing within a fold. Comparisons included native versus non-native English readers, two different programs, and early versus late portions of recordings.

Simple fixation-count and spatial-spread baselines provided essential reference points. More elaborate descriptors summarized returns to previously viewed locations or represented attended regions as graphs.

When scrambling helps—and exposes a confound

Shuffling fixation order improved a descriptor combining mean fixation duration and mean movement distance on three problems. For distinguishing the two programs, AUC rose from 0.546 to 0.778.

The explanation is not that disordered looking is better. Before shuffling, movement distance measures steps along the actual gaze sequence. Afterwards, it approximates distances between randomly paired locations—a measure of spatial spread. The transformation changes what the feature measures, revealing a stronger layout-related distinction.

A separate result illustrates the danger directly. When both programs were pooled, spatial spread alone distinguished correct from incorrect answers with AUC 0.702. Most answers to one program were incorrect, while most answers to the other were correct; their layouts also differed. Restricting the analysis to one program reduced that baseline’s AUC to 0.472. The pooled result therefore did not establish comprehension decoding independent of stimulus layout.

Similar scores, different sensitivities

For native versus non-native English readers, a descriptor summarizing returns to viewed locations reached AUC 0.657, nearly matching the simple duration-and-movement descriptor’s 0.653. With the linear classifier, the return-based descriptor was sensitive to order and coarse, line-scale jitter, but not clearly to the finest tested jitter.

However, order sensitivity disappeared with a second classifier using boosted trees, while one coarse-jitter effect remained. This limits any claim about reading mechanisms: sensitivity belongs to the descriptor and learning algorithm together. Between-reader comparisons also cannot isolate language background from other group differences.

Promising diagnostics, exploratory evidence

Uncertainty estimates resampled participants and retrained models. In the tested simulation designs, intervals covered their targets in 99.6% of 2,720 cases—conservative performance, not a guarantee for other settings. Intervals could be wide, and the many comparisons were not adjusted for multiple testing.

The authors also checked scores against shuffled-label reference distributions because some evaluations produced no-signal AUCs below 0.5. Longer runs preserved most selected interpretations, but not every borderline result.

The practical contribution is a diagnostic discipline: compare complex models with simple baselines, test suspected stimulus confounds directly, and distinguish a pipeline’s predictive sensitivity from evidence about human thought.