When AI coding models spend their entire output budget reasoning

When reasoning crowds out the answer

Explicit reasoning models produce a reasoning trace before their final response. For code generation, both stages draw from the same output allowance: tokens spent exploring an algorithm are no longer available for writing the implementation.

The authors call the resulting failure reasoning-trace overflow. Their definition is specific: generation reaches its output limit while still in the reasoning segment, without transitioning to a final answer. This differs from a long explanation that eventually produces a solution. Under the study’s evaluation protocol, an unfinished reasoning trace has no final answer to assess.

Seven models under a shared limit

The researchers evaluated seven open-weight models across six benchmark settings, giving every generation a 32k-token output budget. Their selection included Qwen3 models with 8, 14 and 32 billion parameters, alongside Llama-R1-8B, OLMo3-7B and two NVIDIA models that the paper calls OpenReason-7B and CodeReason-7B.

The benchmarks covered small function-writing tasks, practical Python programming, competitive programming, and short-answer questions about code inputs and outputs. Using greedy decoding, the researchers generated one response per task and measured pass@1: whether that single response satisfied the benchmark’s correctness checks.

Overflow varied substantially. It was generally uncommon on the simpler EvalPlus function-writing tasks, although OpenReason-7B still overflowed on 27.7% of them. Harder competitive-programming benchmarks produced much higher rates for several models. Across some model–benchmark pairs, more than half of responses exhausted the entire allowance while still reasoning.

Short answers did not eliminate the problem. Several models overflowed frequently when asked to predict inputs for short Python functions. The authors interpret this as evidence that overflow is not simply a consequence of needing to write lengthy code.

Repetition explains much, but not all

Across overflowed generations, 94% contained a detected reasoning loop. The detector looked for sustained periodic repetition spanning at least 2,000 characters and three complete cycles.

That association does not make all lengthy reasoning equivalent to looping. Among responses that used more than 16k reasoning tokens but nevertheless reached an answer, 99% contained no detected loop. The findings therefore distinguish repetitive failures from extended reasoning that successfully ends.

Within the Qwen family, larger models overflowed less often. The authors suggest that these models might need less extended reasoning or be better calibrated about stopping, but the comparison does not establish which explanation is responsible.

Turning reasoning off is not always better

The team also forced models to begin directly in the answer segment. This disabled the explicit reasoning trace, not latent internal computation; final responses could still contain reasoning-like text.

Direct answering was often competitive on EvalPlus and BigCodeBench. For overflow-prone models, reaching the answer stage could outweigh the benefits of additional reasoning.

However, stronger models gained substantially from reasoning on harder tasks. Qwen3-32B’s pass@1 increased by 25 percentage points on LiveCodeBench and 30 points on CodeContests when reasoning was enabled. Across the two CRUXEval settings, reasoning-enabled performance for the Qwen models averaged 34 percentage points above direct answering. These results argue against a blanket recommendation to disable reasoning.

Soft requests versus enforced transitions

Prompt instructions offered inconsistent control. For Qwen3-8B, an instruction to briefly consider key implementation concerns reduced average overflow from 21.4% to 3.6% while retaining similar accuracy. Yet every steering prompt increased OpenReason-7B’s average overflow. Some brevity instructions also substantially reduced Qwen3-32B’s accuracy.

The harder intervention enforced a transition after a specified reasoning budget, leaving the remaining portion of the unchanged 32k allowance for answering. The researchers tested thresholds from zero to 28k tokens in 4k increments.

Suitable thresholds improved several overflow-prone models. CodeReason-7B gained almost 20 percentage points in average pass@1 over standard reasoning. No single threshold worked best across models and tasks, and enforcing transitions required access to decoding.

What the study leaves unresolved

These are preprint findings from single-shot benchmarks, not a demonstration across coding assistants generally. Larger or closed models, hidden reasoning and multi-turn agents using tools or repository context were outside the study’s scope.

The measurements also depend on models following their reasoning delimiters. The loop detector can miss shorter or semantic repetition, and output-token counts do not directly measure latency or energy. Different tokenizers further complicate cross-model cost comparisons.

Finally, the best prompts and transition budgets were selected retrospectively using benchmark accuracy. Their results show what worked among tested settings, not a deployment-ready method for choosing limits on unfamiliar tasks. The practical conclusion is conditional: preserve useful reasoning, but prevent it from consuming the opportunity to answer.