WoW maps educational training data—and shows where its gains stop

A curriculum map for training data

In an arXiv preprint, Tianze Sun, Yongyao Duan and Sam Chen describe WoW, short for World Knowledge: a Chinese–English educational corpus organized around an Educational Knowledge Graph. The central idea is to make missing knowledge visible before deciding what material to collect next.

The graph contains 76,364 knowledge points across nine broad domains, connected by about 318,000 typed links. These links represent relationships such as prerequisites, equivalent concepts and ideas that are easily confused. Training examples are attached to knowledge points, allowing researchers to retrieve material by concept and identify gaps.

That organization does not mean the corpus is balanced. Computer science accounts for 71.6% of its 188,693 instruction examples. Overall, only 28.5% of graph knowledge points have an attached example. Coverage reaches 88.6% in computer science but just 7.8% in medicine. Those figures measure coverage against the authors’ chosen taxonomy—not completeness against all human knowledge.

Turning documents into reasoning exercises

WoW draws on fourteen collections, including technical articles, textbooks, disciplinary repositories and legal material. The authors report processing about 309,000 documents; the retained instruction examples are grounded in roughly 100,500 source documents.

Their construction pipeline has four phases. First, it cleans and segments documents, using a language model to judge whether adjacent sections belong together. It extracts assessable objects such as central propositions, equations and result tables, keeping closely dependent objects together. Supporting evidence is then retrieved from the same document, including definitions and assumptions needed to solve the intended problem.

Second, the pipeline generates research-style exercises with connected subquestions. Later steps must depend on earlier results or combine additional evidence. The aim is to require analysis rather than merely asking a model to repeat a sentence.

Third, candidates face tests for combining information, transforming it through reasoning and maintaining dependencies between subquestions. Additional scores assess reasoning depth, synthesis and conceptual importance. Finally, suitable problems are converted into different response formats. About 70% remain open-ended; others use single-answer, multiple-answer or table-grounded formats.

Document-level provenance makes the supporting source traceable. It does not, by itself, establish that every generated answer is correct or that every proposed inference follows from its evidence.

Stronger results inside the collection

The researchers fine-tuned three base models—Llama-3.2-3B, Llama-3.1-8B and Qwen3-4B-Base—for one epoch on approximately 183,000 examples. Documents were separated between training and evaluation, preventing examples from the same document from appearing in both.

Before training, the team also corrected conspicuous synthetic-data artifacts. They moved reasoning before the final answer, adjusted how many options were correct in multiple-answer questions and shuffled options where answer references could be safely rewritten. Originally, B was correct in 64% of single-answer items.

Evaluation within WoW measured exact option-set accuracy on 881 selected-response questions, not the predominantly open-ended corpus. Accuracy improved by 15.9 percentage points for the 3B model, 5.2 for the 8B model and 2.8 for Qwen. Paired bootstrap confidence intervals excluded zero for all three gains.

Almost all improvement came from multiple-answer questions, where a model must identify the complete set of correct options. Single-answer changes were small and not statistically significant. The fine-tuned 3B model reached 78.2% overall, above the 8B base model’s 76.7% on this particular evaluation—not evidence that it became generally more capable.

External gains are narrower

Across eight external benchmarks, average accuracy increased by only 0.1–0.3 percentage points. The clearest benefits appeared in the smallest model on science-oriented tests: reported gains included 5.5 points on SciEval, 2.1 on ScienceQA and 1.4 on MoleculeQA. Improvements for the stronger models were generally small and were not marked statistically significant in the results table.

The smallest model also lost accuracy elsewhere, including 1.1–2.6 points on broad English and Chinese exams and 2.3 on the medical control benchmark. The stronger models stayed within 0.7 points of their starting scores on those broad exams and the medical control.

Several limits matter. The experiments do not separately isolate the contribution of graph organization, generated exercises and answer-format corrections. Open-ended reasoning remains unevaluated, while pre-training and multimodal evaluation are deferred to future work despite the full-lifecycle framing. The supplied manuscript also retains unresolved methodological notes, including requests to specify parsing details and filtering thresholds.

This is a preprint, not peer-reviewed confirmation. Its evidence supports a narrower conclusion than universal training improvement: concept-organized educational supervision can improve selected disciplinary tasks, while transfer remains modest and specialization can cost a smaller model accuracy elsewhere.