Can a checklist help an AI model learn from a text-only teacher?

A language-model teacher can provide a written solution without revealing the token-level probability distribution behind it. That distribution carries information about how the model scores possible next tokens; it is not a set of literal instructions. When such information is unavailable, conventional logit-based distillation cannot be applied in its usual form. Can a student still benefit from the teacher’s text, and what kind of feedback helps it improve?

Zhepei Hong and colleagues examine those questions with Rubric-based On-Policy Distillation, or ROPD. “On-policy” means the student generates answers using its current policy, and training feedback is evaluated on those student-generated responses. The aim is to keep the learning signal connected to the behavior the student currently produces—not only to answers supplied in advance by the teacher.

For each problem, ROPD collects several teacher responses alongside a group of student responses. A component called the Rubricator uses both sets to write weighted criteria. A separate Verifier checks each student answer against the same criteria, without being told the answer’s source. The weighted pass scores then provide rewards for updating the student through an optimization method called GRPO.

The two sources of text have different roles: teacher responses indicate qualities of strong solutions, while student responses help reveal which criteria distinguish the student’s current attempts. Using one rubric for the group also gives the responses a shared basis for comparison. The paper’s examples include criteria such as explicitly justifying intermediate steps and avoiding arithmetic errors; these illustrate rubric items, rather than proving that every generated rubric uses them.

What the experiments found

In a matched comparison, the student was Qwen3-4B and the open-weight teacher was Qwen3-30B-A3B. ROPD used teacher text, while logit-based methods used token-level information from the teacher. ROPD scored higher than the tested logit-based methods on four competition-math benchmarks: AIME 2024, AIME 2025, and two Harvard-MIT Mathematics Tournament tests. On AIME 2024, it reached the best accuracy reported for the standard logit-based method with about 1,600 training samples, versus 15,400 for that method—about 9.6 times fewer in this particular setup. This is a comparison within the study, not a general efficiency guarantee.

The researchers also compared feedback signals against answer correctness on a fixed pool of AIME 2024 responses. Rubric rewards had an area under the curve (AUC) of 0.90, while teacher log-likelihood had an AUC of 0.35. AUC measures how well a signal ranks correct responses above incorrect ones; 0.5 corresponds to chance ranking. Here, the reported likelihood score ranked below that reference on this analysis pool. The authors suggest that likelihood can favor fluent but flawed reasoning over correct solutions that differ in style. This result concerns the specific pool and comparison, not teacher likelihood in general.

A criterion-level analysis found that ROPD improved more initially failed criteria and regressed on fewer previously passed criteria than logit-based supervision. In the reported cell transitions, 50.0% of initially failed criteria improved under ROPD, compared with 29.3% under LOPD; regression rates were 6.2% and 15.9%, respectively. Ablations also measured the effects of changing rubric construction: using only one teacher response caused a large performance drop, while removing shared rubrics or blind verification also reduced the reported score. These comparisons help identify which design choices mattered in the tested setup, but do not establish that the same effects will hold elsewhere.

Limits and open questions

The main matched comparison focuses on competition mathematics, one student model and one teacher configuration. The paper also reports results on science, medical and instruction-following evaluations, and a test with one alternative student architecture. Those additional results broaden the evidence, but do not show that performance will generalize across tasks, models or deployment conditions. The authors specifically identify open-ended tasks and broader student families as areas for future work.

Automated rubrics and verifiers can be mistaken or exploited. The authors report checklist-exploitation cases—such as keyword stuffing or formatting tricks—in fewer than 2% of rollouts, mainly early in training. That figure describes their experiments; it is not independent confirmation that other failure modes are absent. The paper also cautions that rubric generation may carry forward a teacher’s biases, so explicit criteria are not automatically fair or reliable.

This work is an arXiv preprint, not peer-reviewed validation. Its results support the narrower conclusion that rubric-based, text-only feedback performed strongly in the settings the authors tested. They do not show that rubrics are generally interchangeable with token-level probability supervision, or establish safety performance. Further evaluation is needed to determine how well the approach transfers and how its criteria should be audited.