Predicting the next lesson is not the same as planning a good learning path

Two different questions about recommendations

Learning-path recommendation involves more than choosing a relevant video or exercise. A sequence may also need to introduce supporting concepts before material that depends on them. But the relationships used to enforce that order can themselves be uncertain: two topics appearing consecutively in a course does not prove that one is a prerequisite for the other.

In this preprint, M. M. Golam Hafiz and colleagues examine immediate prediction, uncertain prerequisite structures and the behavior of longer generated paths as separate evaluation problems. Their findings do not establish that their proposed graph model is the best recommender. Instead, they show why success on one measure should not stand in for evidence on another.

This account is based on the supplied manuscript of arXiv:2611.09345v1. Its PDF identifies version one and the date 6 November 2026. The accompanying metadata gives an exact submission timestamp of 2026-11-06T11:42:03Z, but no version-specific arXiv submission-history or API record was supplied or independently retrieved. That timestamp therefore remains unverified; the PDF establishes only the revision and calendar date.

Building and testing an uncertain knowledge map

The main experiment used MOOCCubeX, a collection of online-course resources. Development involved 3,187 courses, followed by a frozen evaluation on 541 separate courses containing 37,747 next-resource transitions. Model choices were fixed before that final evaluation.

The task was to predict the next authored resource from the resources already encountered in a course. It was not personalized to an individual learner’s mastery, preferences or answers. Matching the course’s existing order was the prediction target, not proof of educational benefit.

The proposed model combined concept representations with information passed along prerequisite connections. The researchers automatically checked and revised those connections, assigned confidence levels and compared several ways of using them. Hard constraints excluded resources with unmet prerequisites; soft constraints supplied penalty information without excluding candidates; a hybrid policy combined the two.

These checks were automated, not expert validation of the prerequisite map. The study also tested missing, added, reversed and confidence-perturbed connections.

Simpler alternatives won the immediate prediction task

On the frozen MOOCCubeX evaluation, XGBoost, a tree-based ranking model, achieved the strongest observed mean reciprocal rank, or MRR, at 0.9048. The proposed graph model scored 0.7707. MRR rewards placing the correct next resource near the top of a recommendation list; it does not measure how much someone learns.

A linear ranker and a neural model without graph propagation also exceeded the proposed model on this measure. In a separate controlled development comparison, the version with no prerequisite graph achieved the highest immediate MRR among the graph variants.

A feature audit revealed another important dependency. Masking permitted title-derived features in the trained model reduced development MRR from 0.7668 to 0.0076. This indicates strong reliance on title and title-path information. It does not, by itself, demonstrate data leakage: explicit course identity, resource-order and target-derived shortcuts were excluded. Nor was this masking experiment equivalent to training a new model without those features.

Longer paths revealed different weaknesses

For path generation, each recommended resource became part of the input for the next recommendation. The researchers measured properties including concept coverage, redundancy, prerequisite violations and successful termination.

However, this evidence was substantially smaller than planned. The protocol selected 60 held-out courses, but completed outputs contained only 40 model-and-method results across 12 distinct courses. Graph-model beam search, which considers several possible continuations, had completed results for just four courses.

Among the completed greedy paths, the graph model covered fewer goal concepts on average than the concept-continuity and random-valid baselines. All reported path summaries had zero violations under the active prerequisite policy. That combination illustrates the distinction: obeying a supplied map does not ensure broad coverage, and it cannot establish that the map is pedagogically correct. These limited comparisons also do not establish how the strongest immediate ranker would perform on full paths.

External evidence, with boundaries

An independent experiment used Junyi learner exercise histories. Its structural information was a topic hierarchy, not an explicit prerequisite graph, and models were trained within Junyi rather than transferred unchanged from MOOCCubeX.

A Markov baseline achieved the highest observed MRR, 0.4453 versus 0.3954 for the proposed graph model. The graph model nevertheless placed the target within its top ten more often, showing that rankings depended on the metric. Junyi rollouts also assumed correct answers when updating simulated learner histories.

The study measured neither learning gains nor outcomes from an educational intervention, and included no completed expert validation of prerequisite edges. Concept mapping was incomplete, and prerequisite relations covered only part of the evaluation space. Its practical conclusion is methodological: prediction accuracy, structural consistency and longer-path behavior need separate evidence. None alone demonstrates that a recommendation system helps people learn.