What Movie Recommenders Miss When They Never Ask

Recommendation systems usually infer what people want from what they and others have already rated. But could a platform learn something important simply by asking what someone expects from a particular movie?

In a November 2026 arXiv preprint, Zihan Chen, Lei Nico Zheng and Di Zhu report that MovieLens members’ expectations predicted their later ratings more accurately than four algorithmic benchmarks. The finding concerns a selected group of recorded ratings, not a demonstration that people always know their preferences better than machines.

A prediction before the rating

Between March 2023 and May 2024, MovieLens homepage prompts asked members whether they had seen a movie. Those answering “not seen” could predict their rating on the platform’s half-star-to-five-star scale and report their certainty.

The researchers identified 26,020 distinct member–movie expectations. Their main analysis used just 551 pairs, involving 277 members and 389 movies, followed by a recorded rating at least one day later. They excluded known earlier ratings or reported viewings. The median gap between expectation and rating was 58 days.

That delay is measured between records, not necessarily between prediction and viewing: MovieLens does not stream movies and generally does not know when members watched them.

The benchmarks were MovieLens’s own prediction, two collaborative-filtering models and a model combining machine forecasts with movie and member information. Collaborative filtering learns from patterns in users’ ratings: one model represented shared taste patterns, while the other used a member’s ratings of similar movies.

The researchers trained their recommenders using up to approximately 33 million ratings. Crucially, they incorporated each member’s latest ratings available before the expectation, rather than relying only on monthly snapshots.

Better at predicting a rating, less clearly better at ranking

Across the 551 pairs, expectations correlated 0.69 with later ratings, compared with 0.34 for MovieLens and 0.54–0.59 for the trained recommenders. Correlation measures how closely predictions and outcomes move together; it is not the percentage of predictions that were correct.

Members’ guesses also missed their later ratings by less: an average absolute error of 0.61 stars, compared with 0.72 stars for the strongest machine benchmark.

The advantage remained after accounting for a movie’s general appeal and a member’s tendency to give higher or lower ratings. The authors interpret this as evidence that expectations contain information about the particular person–movie match.

But predicting a rating and ranking movies are different tasks. Among 77 members with at least two differently rated movies, expectations ordered movie pairs correctly about 72% of the time. MovieLens achieved 56%, while the trained recommenders achieved approximately 64–67%.

The differences between people and the trained recommenders were not statistically distinguishable from zero. That does not prove equal performance, but it limits claims of human superiority for ranking.

What asking added

Adding the current expectation to the combined machine information reduced out-of-sample mean squared error by 26%, from 0.921 to 0.685. This measure gives larger prediction errors greater weight. Evaluation kept each member’s movies together in held-out groups, and the improvement also appeared when training on earlier expectations and testing on later ones.

The specific question mattered. Tested summaries of members’ earlier expectations, including summaries emphasizing similar movies, did not provide a statistically detectable improvement or replace the current answer. Certainty added little beyond the expected rating.

The authors frame these results as consistent with dispersed knowledge: users may hold relevant information that recorded behavior misses. However, the study cannot isolate private self-knowledge from other information, such as movie details omitted by the benchmarks.

Where the conclusion stops

Only 2.1% of distinct expectations entered the main sample, and 98.8% of all homepage prompts went unanswered. Members chose whether to respond, what to watch and what to rate.

Reweighting analyses supported the main comparison, but depended on an assumption: after accounting for measured variables, recording a rating must not depend on the rating itself. The better-balanced analysis also excluded very obscure movies.

Machines matched or exceeded people in a separate sample of older, heavily rated movies that members had previously rated despite reporting them unseen. The study also did not test sequential, deep-learning or language-model recommenders.

The authors propose asking about specific candidates, making certainty optional, evaluating ranking among actual recommendation candidates and giving benchmarks current user histories. These are design proposals, not demonstrated improvements in viewing satisfaction. A randomized experiment would be needed to establish whether asking—and using the answer—actually improves recommendations people act on.