A wider view of gaze may help AI rank language-model responses

A person reading a sentence does not take in only the word directly under their gaze. Visual information can also come from nearby text, and the region available to readers is not necessarily the same on both sides of a fixation. That raises a practical question for AI: if a system uses gaze-related signals to learn which chatbot answers people prefer, should it treat attention as a point—or as a spread-out region?

A reward model is a component used in some language-model training approaches. It learns to score responses so that preferred answers receive higher scores than less-preferred ones. Such a model usually gets preference labels—such as which of two answers was chosen—but not a direct record of how a person read each answer. Eye-tracking features may add information about that process. Earlier gaze-augmented approaches have represented those features at the token level, attaching a signal to the word or token associated with a fixation.

The authors explore a different representation. They build on GazeReward, which uses a pretrained model to predict reading-related gaze features from text. Rather than collecting new eye-tracking measurements for every preference example, the system generates token-level gaze features, including total reading time. The new method focuses on total reading time, or TRT, and redistributes each token’s predicted TRT across neighboring tokens.

The redistribution uses an asymmetric, Gaussian-shaped curve: influence is greatest near the original token and fades with distance, but can fade at different rates to the left and right. The two spread parameters are learned during reward-model training. The resulting features are then added to, or combined with, the text representation before the model learns from chosen-versus-rejected response pairs. The authors tested two Llama-3 reward-model variants on OpenAssistant Conversations, using two ways of combining gaze and text features. They also tested one configuration on HelpSteer2 and UltraFeedback. Each condition was run with six random seeds, and performance was measured as the share of preference pairs ranked correctly.

The redistributed-gaze model had higher mean accuracy than its corresponding token-local gaze baseline in every reported setting. On OpenAssistant Conversations, for example, accuracy rose from 68.38% to 74.73% for one model in the concatenation setup, and from 69.23% to 74.81% for the other. In the additional tests, the reported figures rose from 59.48% to 62.64% on HelpSteer2 and from 65.65% to 74.20% on UltraFeedback. These results apply to the study’s particular datasets, models, and training setup; they do not establish that the method will improve every reward model or preference task.

The learned spread often extended farther to the right than to the left, matching the direction expected for people reading left to right. But this was not universal: HelpSteer2 was the exception, with a wider learned spread to the left. The authors suggest that differences in how chosen and rejected responses could be selected may matter, but they do not establish that explanation. An ablation offers another caution: a learnable symmetric spread also performed well. Its mean accuracy was close to the asymmetric version, so the gain cannot be credited to left–right asymmetry alone.

The results support a limited claim: smoothing predicted reading-time signals across nearby tokens was useful for preference prediction in these experiments. They do not prove that the learned curve is a direct measurement of human attention, or that gaze caused people to prefer particular answers. The gaze features were predicted by a model, not collected as live eye-tracking measurements from the people who supplied the preferences. The authors describe the Gaussian as a simplified approximation: attention can depend on reading skill, text difficulty, task, and context, and may not always decline smoothly with distance. Their method also uses token distance, which is not the same as physical letter spacing, and omits other gaze features that might carry useful information.

The work is presented as an arXiv preprint. The findings should be treated as provisional; the supplied source does not establish independent replication or validation beyond the experiments described. For now, the paper offers a modeling idea—not evidence that a reward model reads or understands text as a person does.