A New Training Recipe for One-Step Image Generation

The preprint Trajectory-Regularized Distribution Distillation for One-Step Generation, by An Zhao and colleagues, addresses how to train flow-matching models for generation with very few sampling steps. It presents trajectory-regularized distribution distillation, or TRD, which combines supervision based on a teacher’s path with supervision based on the student’s generated distribution.

Two training signals

The authors describe two broad approaches to distilling flow-matching models. Trajectory alignment trains a student using the teacher’s probability-flow ordinary differential equation (ODE), a mathematical description of the teacher’s generation dynamics. The abstract says this approach’s performance can degrade when the sampling budget is very small.

Distribution matching instead directly optimizes the distribution generated by the student. The abstract notes a challenge: estimating the gradient of the reverse Kullback–Leibler (KL) objective typically requires a separate network to track the student’s evolving score. KL divergence is a way to measure a difference between probability distributions; here, the abstract refers specifically to a reverse-KL gradient, without detailing its target or estimation procedure.

TRD aims to combine these signals. It uses a “momentum-mixture policy” predicted in a single student forward pass. According to the abstract, this policy serves as a one-step generator, estimates the student’s marginal velocity for distribution matching, and defines a time-dependent velocity field supervised along the teacher’s ODE trajectory. The authors say this removes the need for a second trainable network in distribution matching.

In broad terms, the method is intended to use information about both the teacher’s trajectory and the student’s evolving output distribution. The abstract describes the roles of the proposed policy, but does not provide enough detail to reconstruct the algorithm or assess how each component contributes.

What the authors report

The authors tested TRD with SANA-1.6B and Stable Diffusion 3.5. They report that it outperformed representative consistency and distribution-matching baselines in generation quality and training efficiency for both one-step and few-step generation.

The abstract also reports a trade-off in sample diversity. That qualification matters because quality and diversity describe different aspects of generation: strong results on one do not establish that the range of outputs is unchanged. The abstract does not say how large the trade-off was, how diversity was measured, or whether it varied between models or settings. It also gives no numerical results or named evaluation metrics, so readers cannot judge the magnitude of the claimed improvements from the available evidence.

What remains uncertain

This account is based on the abstract rather than the full paper. It therefore cannot establish the experimental setup, the specific baselines, the measures used for quality, diversity, and training efficiency, or the results across individual conditions. The abstract also does not establish how TRD would perform with other models or outside the reported experiments.

The source is an arXiv preprint, and the supplied record does not establish its peer-review status. The findings should be understood as the authors’ reported experimental claims, not settled evidence. The proposal is a way to combine trajectory-level and distribution-level supervision, but the full paper and independent replication would be needed to assess its benefits, trade-offs, and broader applicability.