Speed-Variation Distillation for Generative Modeling

TL;DR

Standard diffusion denoising trajectories pass through corrupted, unrecognizable intermediate states. We distill generation into a variation-reducing reverse process that preserves recognizable images on the data manifold at every single step—enabling early termination and a controllable speed–variation trade-off.

Abstract

While diffusion models excel at image generation, their denoising trajectories pass through corrupted intermediate states far from the image manifold. We ask whether generation can instead proceed through recognizable images throughout the entire trajectory. We introduce a generative framework built around a variation-reducing degradation process that contracts diverse images toward a shared prototype while preserving recognizable intermediate states. Because such manifold traversal lacks a simple analytical form, we construct the trajectories using generative-prior-based image morphing. We then distill the reverse process into a diversification operator that transforms the prototype into increasingly varied samples. To resolve the one-to-many ambiguity of the reverse process, we associate each trajectory with its initial noise latent and apply perturbations to improve robustness against accumulated errors. At inference, every step yields a recognizable image, allowing early termination and enabling a controllable speed–variation trade-off.

Method Overview

Figure 2: Method Overview. (a) Generate images with a pretrained LDM, select a prototype, and construct variation-reducing trajectories using image morphing. (b) Train a network conditioned on timestep and initial noise to distill the reverse diversification process. (c) Inference loop generating diverse samples from a prototype.

Variation-Reducing Process

Instead of corrupting images into random Gaussian noise, our degradation contracts diverse images toward a shared prototype while keeping all intermediate states strictly on the image manifold.

Prior-Based Trajectory Alignment

Because manifold traversal lacks a simple closed form, we leverage generative-prior image morphing to map smooth, semantically continuous trajectories between prototype and target samples.

Noise Latent Association

To resolve the one-to-many mapping in the reverse diversification direction, each trajectory is associated with an initial noise latent and perturbed to prevent error accumulation.

Speed–Variation Trade-Off

Inference can be terminated at any intermediate step. Stopping early yields high-quality recognizable images in fewer steps, allowing flexible trading between generation speed and sample diversity.

Interactive Trajectory Explorer

Compare intermediate generation states across timesteps (Steps 1 → 50). Watch the auto-slide or manually drag the slider!

Sampling Timestep Progression

Step 1 of 50

Step 1 (Start) Step 10 Step 20 Step 30 Step 40 Step 50 (Final)

Speed-Variation Distillation (Ours) Recognizable Manifold

Step 1: Instantly yields a crisp, recognizable prototype structure on the manifold.

Standard Diffusion (Baseline) Corrupted Noise

Step 1: Pure random Gaussian noise (far from image manifold).

Quantitative Quality vs. Diversity Analysis

Key Empirical Insights

Ideal Manifold Tracking: Our method (blue solid curve) closely tracks the reference Dataset distribution curve (red dashed curve) across the entire generation trajectory.
High Quality at Low Diversity: While baselines ($\hat{x}_0$ orange, $x_t$ green) start at low quality values (-2.0 to -2.3), our trajectory initiates at peak image quality right from the first step.
Controllable Diversity Trade-Off: Stepping forward progressively expands sample diversity (from 1.0 to 8.0) while preserving superior image quality throughout traversal.

BibTeX Citation

@inproceedings{seripanitkarn2026speed,
  title     = {Speed-Variation Distillation for Generative Modeling},
  author    = {Seripanitkarn, Sukit and Tawatdamrongkrit, Phonphrm and Suwajanakorn, Supasorn},
  booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
  year      = {2026}
}