Teaching AI When to Stop Reasoning

The problem is knowing when enough is enough

Long reasoning outputs can contain useful calculations, revisions and checks. They can also repeat work that has already settled a problem. In an unreviewed preprint, Muyao Yuan and colleagues describe a training method called LoCo, short for Local Compression, that targets this distinction.

Their central proposal is to compare different continuations from exactly the same point in a solution. Instead of teaching a model that shorter responses are generally preferable, they try to teach it when a shorter finish is sufficient.

The authors report reductions in generated completion length of 41.7%–45.4% across five mathematics benchmarks and 23.4%–36.3% across three coding benchmarks. Accuracy was broadly maintained or improved relative to the original model. These are experimental results for a particular model and evaluation setup, not evidence that all AI reasoning can be shortened safely by the same amount.

Comparing two endings to the same reasoning

LoCo starts with mathematical solutions generated by the model being trained, DeepSeek-R1-Distill-Qwen-7B. The researchers sampled three solutions per question from a training pool containing 14,027 mathematics questions, selecting the shortest correct solution where available.

They divided each reasoning trace into steps at paragraph boundaries. To find possible compression points, they measured how strongly the model favored its most likely next token, averaged over each step, and how much that measure changed between neighboring steps.

Steps combining relatively low predictive confidence with high fluctuation became candidates for further inspection. This is a screening signal, not a direct measurement of whether the model understands a problem or has finished solving it.

A larger model, Qwen3-30B, then reviewed the candidates. It distinguished redundant reasoning from useful progress and final-answer wrap-ups. Only candidates judged redundant advanced to the next stage.

At each retained point, the researchers preserved everything already written and asked the original model to generate eight alternative continuations. They selected the shortest continuation that completed the required format and produced a final answer symbolically equivalent to the reference answer. Training pairs had to save at least 300 tokens, with no more than three retained per question.

The model was then trained using direct preference optimization, or DPO: a method that increases its preference for a selected response over a rejected alternative. Here, the preferred response was the shorter verified continuation; the rejected response was the original, longer ending. Both shared the identical starting context.

Shorter outputs, with some accuracy gains

The mathematics results illustrate why length and accuracy need to be assessed together. On MATH-500, average completion length fell from 4,015 to 2,243 tokens—chunks of text processed by the model—while accuracy changed from 91.88% to 91.80%, a slight numerical decrease.

On AIME 2024, the authors report a larger improvement: average length fell from 13,493 to 7,364 tokens, while accuracy rose from 50.83% to 54.17%.

The same mathematics-trained model also produced shorter coding solutions. On MBPP, accuracy remained at 84.05% while completion length fell by 36.3%. On CodeContests, accuracy increased from 18.48% to 20.61%, with 23.4% fewer tokens.

The researchers sampled eight completions per problem. They also measured the area under an accuracy-versus-token-budget curve, which summarizes performance across different output allowances. LoCo had the highest reported value among the compared methods on all three coding benchmarks.

Why the stopping point matters

The authors tested variants that changed individual parts of the method. Comparing complete short and long solutions, without a shared intermediate reasoning state, produced much smaller length reductions on the three mathematics benchmarks used for this comparison.

Randomly chosen or evenly spaced compression points also shortened outputs substantially, but generally preserved accuracy less well than LoCo. These experiments support the authors’ argument that where compression happens matters, not just whether training rewards brevity.

What remains unproven

The study evaluates one base model, with training restricted to mathematical questions. Transfer to three coding benchmarks is encouraging, but does not establish reliability across other model families or open-ended tasks.

Verification also has a specific meaning here: mathematical continuations were checked for final-answer equivalence, not for the validity of every intermediate statement. Candidate selection depended partly on another model’s judgment.

Finally, the reported efficiency gains concern output token counts, not measured reductions in latency, energy use or deployment cost. Data construction itself required additional generation and judging. The preprint therefore offers evidence for a targeted training approach to shorter reasoning, rather than a demonstrated end-to-end cost saving.