Teaching smaller AI models with corrections they can follow

Correct answers are only part of the lesson

Training a smaller language model on solutions written by a stronger one is called reasoning distillation. But the teacher’s preferred solution may contain transitions the student struggles to generate independently. The student can then drift away from the examples it encountered during training.

Yijun Zhu, Jianxin Wang and Chengchao Shen address this mismatch with Failure Profiling and Reachable Trajectory Selection, or FPRT. Their paper was posted to arXiv on November 6, 2026, and its landing record states that it was accepted to NeurIPS 2026. The reported experiments support a specific training-data strategy, not a general demonstration that models reason reliably.

Diagnose the pattern before correcting it

The pipeline first samples five answers from the student for each training question. It measures how often the student is wrong and how strongly its incorrect answers concentrate on one particular result.

Questions with at least three wrong answers are classified as either “stubborn” or “inconsistent.” Stubborn failures have at least 80% of their incorrect answers concentrated on the same wrong result. Inconsistent failures are more scattered. Questions with only one or two wrong answers are classified as “sporadic.” Questions answered correctly in all five attempts are excluded from the corrective pipeline.

These labels describe sampled behavior; they are not direct measurements of an internal misconception. The researchers nevertheless use them to choose different teaching strategies.

For stubborn failures, the teacher proposes corrections that approach the apparent error from different angles. For inconsistent failures, it supplies structured problem-solving frameworks. For sporadic failures, it attempts a small local repair while preserving correct earlier work.

The teacher, DeepSeek-V3.2 in the main experiments, receives a representative incorrect solution and generates five candidate corrections. Each contains a short diagnosis, a preserved correct prefix where possible, and a continuation to the answer. Candidates whose final answers fail the ground-truth check are discarded. This checks the outcome, rather than independently verifying every intermediate step.

Score the first corrective steps

FPRT then asks which surviving correction best fits the student’s own output probabilities immediately after the identified error boundary.

Its branching point reachability score averages the student’s token log-probabilities over the first eight sentences of the corrective continuation, conditioned on the question and preserved prefix. Sentence boundaries are extracted using newlines, with punctuation-based splitting as a fallback.

The motivation is that later steps may become relatively easy once a correct context has already been supplied. Scoring an entire solution could therefore obscure the difficult transition away from the original error.

The score is a selection proxy, not a guarantee that the student will reproduce or learn a solution. The paper explicitly distinguishes it from a calibrated estimate of the probability of the complete reasoning trace.

Selected examples are used for standard supervised fine-tuning. Most teach correction from an erroneous attempt; 10% instead pair the question alone with a complete corrected solution, supporting question-only use at evaluation.

Better averages, with a clear exception

The researchers trained Llama-3.1-8B-Instruct and DeepSeek-Math-7B-Instruct, evaluating answer accuracy on GSM8K, MATH, SVAMP, ASDiv and MAWPS. The latter three were treated as out-of-distribution tests relative to the training sources.

With 88,900 training examples, FPRT achieved five-benchmark averages of 79.7% for Llama and 80.6% for DeepSeek. These were means across three fine-tuning seeds, exceeding LEMMA, the strongest comparison with a reported five-test average, by 2.4 and 1.8 percentage points respectively.

That average lead did not mean winning every benchmark. On MATH, FPRT scored 46.2% and 48.9%, below LEMMA’s 48.8% and 50.7%. Its advantages were more pronounced on the out-of-distribution tests.

Component-removal experiments also supported both design choices. In the smaller Llama configuration, replacing failure-specific prompting with uniform minimal corrections reduced average accuracy from 79.0% to 72.9%. Replacing localized scoring with whole-correction scoring reduced it to 77.3%.

What remains uncertain

The method depends on a strong teacher to generate useful candidates and locate the first error. A manual audit found accurate boundaries in 44 of 50 inspected DeepSeek traces, leaving room for localization mistakes.

Five samples also provide only a coarse picture of each question’s failure pattern. The authors did not systematically explore the cost–benefit trade-off of larger sampling budgets. Smaller final training corpora should therefore not be confused with lower total data-generation cost.

Finally, the evidence concerns mathematical benchmarks and two student backbones. It supports matching corrective examples to the learner in this setting, rather than establishing effectiveness across other tasks.