A discrete-time bridge between deterministic diffusion and flow matching

Deterministic generation presents a mathematical awkwardness for the standard variational treatment of diffusion. A generator that always returns one particular next state does not fit neatly into the probability comparisons used to train a stochastic diffusion model. In their preprint, Rares Dolga, Harshad Dahake and David Barber address that problem and derive a connection to flow matching directly in discrete time.

The contribution is theoretical: a modified training objective, an equivalence under specified choices, and a convergence theorem with explicit assumptions. Its significance is a shared explanation for these deterministic generators, rather than a demonstrated improvement in image quality or sampling speed.

Making deterministic steps comparable

In the standard diffusion setup described by the authors, a forward process progressively corrupts data with Gaussian noise. A stochastic generator learns to reverse that process. The variational training objective compares backward transitions using Kullback–Leibler, or KL, divergence.

Trouble arises when a transition becomes deterministic. Its probability mass sits at a single point, represented mathematically by a Dirac delta. The per-step KL comparisons needed by the usual objective are then ill-defined. The authors contrast their approach with DDIM, which they describe as training a stochastic generator before making its sampling transitions deterministic.

Their proposed Degenerate Diffusion Model instead uses deterministic generating steps throughout. Its forward construction pairs a data point with one Gaussian noise sample and interpolates between them, reusing that same noise across every timestep. Generation starts from random Gaussian noise, but each subsequent update is deterministic. Deterministic steps therefore do not mean that all generated samples are identical.

The mathematical repair uses an existing idea called spread divergence. Both distributions being compared are convolved with the same Gaussian kernel. Point masses become Gaussian distributions with matching covariance, making their divergence finite. The resulting objective reduces to weighted mean squared errors between the target next state and the generator’s prediction.

Crucially, this smoothing defines the training comparison. It does not require injecting fresh randomness into the deterministic sampling chain. The authors spread each transition’s output while conditioning on a clean input from the forward process, preserving a tractable, per-step objective.

What the objective actually learns

The generator sees a noisy state, not the original training example that produced it. Several examples can therefore be plausible explanations of the same input. According to the paper, the optimal deterministic prediction is the mean of the corresponding backward distribution, rather than a random draw from its different possibilities.

The authors show that their tractable loss differs from a spread divergence to that backward distribution only by a constant independent of the model parameters. This connects the regression objective to the distribution the generator is intended to approximate.

Their framework also interprets loss weighting. Changing the spread variance changes how strongly each timestep contributes. Assuming a sufficiently flexible network, any fixed positive weighting has the same per-timestep optimum. This is a statement about the ideal objective’s minimiser, not a claim that every weighting produces identical outcomes when training a finite network.

The precise connection to flow matching

Rewriting the generator as a velocity predictor makes the relationship visible. Along each data-to-noise interpolation, the target velocity is simply the noise sample minus the data point. The generator then moves backward using that predicted velocity.

For a linear interpolation schedule and a specified spread variance, the authors obtain exactly the flow-matching loss evaluated on a uniform discrete grid. Their update is also exactly the corresponding Euler sampling step. For smooth nonlinear schedules satisfying the paper’s assumptions, the loss correspondence holds in the limit as the number of steps increases, and the sampling steps agree up to a vanishing correction.

Convergence is not generalisation

The convergence theorem concerns a finite dataset of at least two distinct, equally weighted points. It assumes the generator reaches the exact optimum at every timestep and every input, while the largest interpolation step tends to zero.

Under those conditions, the authors prove that generated samples converge weakly to the empirical data distribution: samples concentrate around the training points, with each point receiving the appropriate equal probability in the limit.

The paper explicitly identifies this as consistency, not an account of a trained network’s generalisation. The result explains how deterministic updates can recover a finite training distribution despite averaging at individual steps. Its central payoff is a discrete-time justification linking that behaviour to flow matching, without requiring continuous-time machinery for the convergence proof.