Knowing When to Stop Is Still a Challenge for Reasoning AI

The cost of deciding to stop

An AI model that has enough information to answer does not necessarily stop generating reasoning. It may check its work, explore alternatives or reconsider a conclusion. Early-stopping methods try to recognize when further reasoning is unnecessary and direct the model to answer sooner.

But deciding when to stop can itself require extra computation. Some methods generate provisional answers and check whether they agree; others inspect uncertainty signals or learned representations inside the model. A shorter reasoning trace therefore does not automatically mean a cheaper overall response.

In their preprint, Quang Hung Nguyen and colleagues audit this trade-off alongside a second question: does stopping preserve a reliable, adequately supported answer? Their findings suggest that efficiency cannot be judged from answer accuracy or reasoning length alone.

Comparing methods on shared trajectories

The researchers evaluated five models: Gemma 4 E4B, gpt-oss-20b, Qwen3.5-4B, Qwen3.5-9B and R1-Qwen3-8B. Eight benchmarks covered mathematics, scientific and broad-domain questions, competitive programming, and factuality or hallucination evaluation.

To compare stopping methods, the team first generated a full reasoning trace for each question. They then replayed that trace, allowing each method to inspect the same sequence of prefixes and select an exit. The stopping methods could not see future tokens or the completed answer.

This protocol applies to methods that leave the reasoning trajectory unchanged until stopping. The audit excluded approaches that actively reshape that trajectory. Its token accounting included reasoning, final answers, verification probes and temporary answers—not just the reasoning retained before an exit.

The authors also introduced Target-Integrated Token Efficiency, or TITE. This combines how much accuracy a method achieves between direct answering and a hindsight reference with its token cost relative to that reference. It is a comparative score, not a percentage reduction in runtime or energy use.

Substantial headroom, uneven practical results

The hindsight reference, called a discrete-budget oracle, selected the shortest tested reasoning budget that produced a correct answer for each question. It used ground-truth answers unavailable to a practical stopping policy.

For gpt-oss-20b on mathematical and scientific benchmarks, this reference reached 89.4% accuracy, compared with 82.4% for full reasoning. On coding, R1-Qwen3-8B reached 51.4% under the reference, versus 38.2% with full reasoning. These comparisons indicate opportunities within the tested budgets, rather than demonstrating an implementable stopping system.

Existing methods captured those opportunities inconsistently. On gpt-oss-20b mathematics and science tasks, Statistical Early Exit matched full-reasoning accuracy at 82.4%, with a TITE score of 0.191. DEER and EntroCut scored lower on accuracy, at 77.3%, but higher on TITE, at approximately 0.38.

No method consistently dominated across settings. Coding and factuality tasks were particularly difficult. Several stopping policies on the factuality benchmarks performed worse than answering without a reasoning trace.

More reasoning was not always better

The team separately sampled exit positions and generated eight independent answers at each position. These experiments tested whether apparent reversals—from correct answers earlier to incorrect answers later—could be explained by ordinary randomness in generation.

Average accuracy generally improved with additional reasoning. Nevertheless, individual trajectories showed reversals exceeding those expected under the authors’ fitted model of nondecreasing correctness with decoding noise. The reported average excess reversal magnitude was about 1.21 percentage points.

This does not mean longer reasoning usually hurts. It means that reaching a correct answer at one point did not guarantee that later exits would preserve it. Among the failed exits examined, stopping too early remained the dominant problem for most methods.

A correct answer is not the whole audit

Forced termination also did not always end reasoning cleanly. The researchers detected explicit thinking markers or softer signals, such as “wait” and “actually,” in the final-answer channel. Rates varied substantially: Qwen3.5-4B showed persistent signals, while gpt-oss-20b showed almost none.

An automated judge also assessed whether retained reasoning supported the final answer. Many early-stopping outputs received lower justification scores than full-reasoning outputs, including some with correct answers. Full reasoning itself showed derivation gaps and contradictions, so these weaknesses cannot all be attributed to stopping.

The authors acknowledge important limits. Detailed reversal tests covered only selected data and models. Lexical markers can misidentify continued reasoning, and automated scoring cannot establish whether visible explanations faithfully represent internal computation. Token costs assumed ideal cache reuse and excluded cache-management overhead, so they do not establish latency, memory or energy savings. The audit’s central message is therefore conditional: useful early stopping must account for verification cost, correctness stability and explanation quality together.