Turning recommendation “what-ifs” into explanations people can read

A streaming service suggests a film. But why that one? One way to probe a recommender is to ask a “what if?” question: which past interactions would have to change for the recommendation to change too? These counterfactual explanations can reveal something about a model’s behavior. In practice, though, the answer may be a long list of edits—not an explanation most people would find easy to use.

A team led by Vojtech Cahlik explores whether large language models can turn those lists into brief descriptions of higher-level preference patterns. Their study, From Counterfactual Edits to Human Narratives, focuses on how to communicate counterfactuals, rather than replacing the method that generates them.

From edits to a narrative

The researchers first generated counterfactual explanations with existing methods. In their setup, a counterfactual is a set of interactions removed from a user’s history so the recommender’s original top-ranked item no longer appears among its top five suggestions. The language model then receives information about some removed interactions and some of the user’s remaining history, expressed as film genres and release years. It generates a natural-language explanation based on the contrast between the two sets.

The researchers tested this pipeline on MovieLens-1M, a dataset of movie ratings, using a variational autoencoder recommender. They compared several counterfactual generators with three language models: Ministral 8B, Gemma 3 12B and Qwen3 8B. The team also tested Direct Preference Optimization, or DPO, a training approach that encourages a model to favor better-scoring responses over alternatives.

Evaluation went beyond whether a counterfactual successfully changed the recommendation. The researchers assessed whether an explanation matched the removed interactions, whether it distinguished those interactions from the rest of the user’s history, and whether it was linguistically acceptable. For final testing, they used classifiers calibrated against separate human-labeled data, rather than relying only on the language-model judges used during optimization.

What the study found

Results varied with both the counterfactual generator and the language model. ACCENT and SPINRec had the highest reported success rates in this experiment and relatively compact counterfactuals. Their summaries also tended to perform well on the study’s semantic measures, although the best results depended on the particular model-and-generator pairing.

DPO improved informativeness in most tested pairings and correctness in all but one, according to the paper. Linguistic-quality scores were generally high, but they varied across settings. The overall message is not that any language model can reliably explain any recommendation: the quality of the starting counterfactual and the choice of language model both matter.

What the scores do—and don’t—show

The evaluation is a step beyond counting edits, but it does not establish that users understand the explanations, find them actionable, or trust the recommendations more. The reported final measures come from classifiers trained on separate human-labeled examples; they are human-calibrated, not direct judgments by users about usefulness in everyday recommendation settings. A fluent explanation can also sound convincing without capturing all the relevant reasons behind a model’s output.

The experiment has a narrow scope. It uses one movie dataset and one recommender architecture, and the language model sees genres and release years rather than film titles. That limits what kinds of preference patterns it can express. The authors also exclude especially large edit sets from successful cases and treat very short ones separately, so the results do not describe every counterfactual their methods might produce.

Finally, a counterfactual is a model-based “what if?”—not proof that changing someone’s real-world viewing history would cause a particular change in their tastes or recommendations elsewhere. The paper describes the explanations as causally grounded in the underlying counterfactual edits, but that should not be confused with establishing real-world causation.

The work is presented as a CIKM ’26 conference paper in the supplied text, while the cited arXiv record is a preprint. The available evidence does not establish the paper’s review status or independently verify its results. Overall, the study offers a promising communication layer for recommendation explanations, but user testing and broader evaluations would be needed to show whether readable summaries are genuinely useful beyond this benchmark.