Training AI drafters to keep more guesses in a row

Speculative decoding speeds up language generation by splitting the work: a small model proposes a block of tokens, and a larger target model checks them in parallel. Tokens are pieces of text, sometimes whole words and sometimes fragments. The aim is to reduce how often the expensive target must run sequentially, while preserving its output distribution through the verification procedure.

But a block is only useful up to its first rejected proposal. Getting a later token right cannot rescue an earlier failure. This makes consecutive acceptance—not simply good predictions at individual positions—the quantity that matters.

In an arXiv preprint, Meng Wei and colleagues introduce D-REAL, short for Drafter Reinforcement of Expected Acceptance Length. It changes the drafter’s training objective without changing its deployed architecture or verification protocol.

Learning which sequences survive

Ordinary token-level training rewards predictions individually. D-REAL instead combines two ingredients: how much the drafter’s probability distribution overlaps the target’s at each position, and how likely all earlier proposals are to survive verification.

Distributional overlap measures the probability mass shared by the two models across possible next tokens. Under the paper’s stochastic verification rule, this overlap is the expected acceptance probability at a given prefix. Weighting it by earlier survival probabilities makes later positions count only to the extent that verification can reach them.

There is another complication: training changes not only the drafter’s predictions, but also which sequences it proposes. The authors derive a gradient estimator that accounts for both effects. A gradient tells the training procedure how to adjust model parameters; here, “unbiased” means its average matches the gradient of the specified mathematical objective. It does not mean the model is free of social bias or factual errors.

That guarantee concerns expected acceptance length for one verification round at a fixed starting context, under stated support and differentiability conditions. It is not a guarantee of the best possible speed over an entire conversation.

In practice, training first samples a draft sequence and records its proposal probabilities. The frozen target then scores the actual drafted prefixes. A second drafter pass computes the training updates using those sampled tokens. Reusing shared context computations helps limit the added cost.

Small token gains, measurable serving gains

The experiments used two five-layer drafters, DSpark and DFlash2, with Qwen3-8B as the fixed target. Each drafter proposed seven tokens per round. Training used approximately 1.3 million prompts with responses regenerated by the target in non-thinking mode.

After six baseline training epochs, the researchers compared one additional epoch of ordinary training, the acceptance-aware method D-PACE, and D-REAL. Evaluation covered ten mathematics, coding and chat benchmarks, using both greedy decoding and stochastic sampling.

The reported acceptance-length metric includes one additional target-generated token per round, not just accepted draft tokens. Across the ten benchmarks, DFlash2’s average rose from 4.88 to 5.10 under greedy decoding and from 4.69 to 4.92 under stochastic sampling. DSpark’s gains were smaller: 5.07 to 5.13 and 4.67 to 4.74, respectively. Continuing ordinary training changed the averages by no more than 0.03 tokens.

Against D-PACE, D-REAL performed better in 37 of the 40 architecture–temperature–benchmark combinations, tied once and trailed twice by 0.01 tokens.

The team also measured serving performance on the GSM8K mathematics benchmark, using SGLang on an eight-GPU Hopper node with 32 concurrent requests. DFlash2’s greedy output throughput increased from 17,634 to 19,288 tokens per second, a 9.4% gain. Across the other tested architecture and sampling combinations, throughput improvements ranged from 2.0% to 6.0%.

Where the claim stops

The extra speed requires extra training work: a D-REAL epoch took approximately twice as long as a baseline epoch in the reported setup. The comparisons therefore match additional epochs, not equal wall-clock training budgets.

The evaluation also uses one target model, seven-token draft blocks and non-thinking generation. End-to-end throughput was measured on one benchmark at one concurrency level; the 9.4% maximum is not a general serving guarantee. The reported tables provide no uncertainty intervals.

Finally, training starts drafts at randomly selected positions in offline target-generated text. Real deployment starts each round wherever the previous round ended, creating a remaining mismatch that the authors acknowledge. This preprint offers evidence for a targeted efficiency improvement—not improved reasoning accuracy or a universal acceleration recipe.