DDIM (Denoising Diffusion Implicit Models), introduced by Song et al. (2020), reformulates the reverse process of diffusion models as a non-Markovian deterministic mapping, enabling high-quality image synthesis with 10–50× fewer function evaluations than DDPM while reusing the same pretrained score network. This breakthrough decouples the generative trajectory length from the number of training timesteps, making diffusion models practically deployable in latency-sensitive applications. Remarkably, DDIM's deterministic sampler defines a bijective encoding that allows exact reconstruction of any real image through inversion — a property that probabilistic DDPM samplers fundamentally cannot achieve.
The core insight
DDIM's key insight is that the denoising score network trained under DDPM's Markovian objective is not intrinsically tied to a Markovian reverse process — the same network can serve as a velocity field for an entire family of non-Markovian generative processes parameterized by a continuous hyperparameter σ. By setting σ → 0, the stochastic reverse SDE collapses to an ODE whose solution can be accurately integrated with large step sizes, enabling fast deterministic generation. The "aha moment" is recognizing that the forward process only needs to have the same marginals q(xₜ | x₀) as DDPM — the conditional structure q(xₜ | xₜ₋₁) can be anything that produces those marginals, freeing the reverse process from Markovian constraints entirely.
How It Works
Forward Process and Marginal Preservation
DDPM defines a Markovian forward process where each step adds Gaussian noise with a fixed schedule. DDIM's key observation is that the marginals of this process are sufficient for training — the joint structure is irrelevant. The forward marginal at any timestep t is:
1
q(xt∣x0)=N(xt;αˉtx0,(1−αˉt)I)
Explanation: Any noisy sample xₜ can be sampled directly from x₀ via a single Gaussian, where ᾱₜ = ∏ₛ₌₁ᵗ αₛ is the cumulative product of noise retention factors and controls the signal-to-noise ratio at step t. Here αₜ = 1 − βₜ for a noise schedule {βₜ}ᵀₜ₌₁, and ᾱₜ decreases monotonically from ≈1 to ≈0 as t → T. This means any xₜ can be written in closed form as:
2
xt=αˉtx0+1−αˉtϵ,ϵ∼N(0,I)
Explanation: This reparameterization expresses xₜ as a deterministic mixture of the clean signal x₀ and independent Gaussian noise ε, which is exactly what the score network is trained to invert.
Non-Markovian Reverse Process
DDIM constructs a family of inference distributions indexed by a vector σ = (σ₁, …, σ_T) ∈ ℝ₊ᵀ. For any such vector, the joint q_σ(x₁:T | x₀) is defined such that all marginals q_σ(xₜ | x₀) remain exactly Eq. [1]. The reverse conditional takes the form:
Explanation: Given access to x₀ and xₜ, this defines the distribution over xₜ₋₁ as a point mass (when σₜ = 0) plus optional stochastic noise σₜ, where the mean interpolates between the clean prediction and the "direction pointing to xₜ." The mean in Eq. [3] has a clean geometric interpretation: the first term is the predicted clean signal scaled to timestep t−1, and the second term is the residual "direction" vector scaled to maintain the correct noise level, adjusted by σₜ² which absorbs the desired stochasticity. At training time, x₀ is unknown, so it is replaced by the model's prediction x̂₀(xₜ, t) = (xₜ − √(1 − ᾱₜ) ε_θ(xₜ, t)) / √ᾱₜ.
The DDIM Sampling Step
Substituting the neural network's noise prediction into Eq. [3] with the clean image estimated as x̂₀, the practical sampling update is:
4
xt−1=αˉt−1x^0 prediction(αˉtxt−1−αˉtϵθ(xt,t))+"direction to xt"1−αˉt−1−σt2⋅ϵθ(xt,t)+noiseσtϵt
Explanation: The sampler decomposes the transition into three geometrically interpretable components: scaling the denoised image estimate to timestep t−1, adding back a controlled amount of the predicted noise direction, and optionally injecting fresh randomness via εₜ ~ 𝒩(0, I). When σₜ = 0 for all t, the process becomes fully deterministic — no fresh noise is injected and the mapping xᵀ ↦ x₀ is a bijection. When σₜ = √((1 − ᾱₜ₋₁)/(1 − ᾱₜ)) · √(1 − ᾱₜ/ᾱₜ₋₁), DDIM exactly recovers the DDPM posterior, completing the unification.
Accelerated Sampling via Subsequences
The defining innovation enabling acceleration is that Eq. [4] can be applied to any subsequence τ = (τ₁ < τ₂ < … < τ_S) ⊂ {1, …, T} with S ≪ T. The effective cumulative noise coefficients ᾱ_τᵢ are directly readable from the pretrained model's schedule. The resulting sampling error scales as O(1/S) for the deterministic sampler — the same asymptotic rate as first-order ODE integration — compared to DDPM's O(1/T) convergence requiring T steps. Formally, Song et al. prove the ELBO objective used to train ε_θ under DDPM is a surrogate lower bound that depends only on marginals, so the pretrained ε_θ remains valid for any subsequence:
5
Lγ=t=1∑TγtEx0,ϵ[∥ϵθ(xt,t)−ϵ∥2]
Explanation: The training objective is a reweighted sum of denoising score matching losses over all timesteps, where γₜ are positive weights; crucially, each term depends only on the marginal q(xₜ|x₀) and not on the Markovian structure of the forward process, justifying arbitrary subsequence sampling.
researcher · Jiaming Song | b. 1997 | DDIM Formulation and Non-Markovian Diffusion Theory
Jiaming Song is a PhD student at Stanford University under Stefano Ermon, whose research sits at the intersection of probabilistic generative modeling and scalable inference. He developed the non-Markovian forward process family and deterministic sampler that constitute DDIM (Song, Meng & Ermon, ICLR 2021), which became one of the most-cited papers in generative modeling with direct descendants in nearly every production diffusion pipeline. A surprising fact: Song simultaneously led development of the score-based SDE framework (Song et al., NeurIPS 2020) that provides the continuous-time theoretical unification of DDPM and DDIM, making him central to two of the three foundational theoretical pillars of modern diffusion models within a single graduate school year.
Figure
DDIM Sampling Geometry: DDPM vs DDIM Trajectories
Figure
DDIM Update Step Decomposition
Intuition
The Mental Model
Imagine you are navigating from a foggy city (pure noise, xᵀ) back to your home (the clean image, x₀). DDPM is like walking through the fog in tiny steps, randomly bumping against walls at each step — you need thousands of steps because each bump (injected noise) sends you slightly off course and must be corrected. DDIM is like owning a GPS: at every position, you can estimate roughly where home is (the x̂₀ prediction), compute the straight-line bearing toward it, and take a large confident stride. You still re-estimate at each checkpoint because the fog distorts your GPS, but you need far fewer checkpoints because you are not fighting self-induced randomness. The σₜ parameter is the "boldness dial" — set to zero, you take only confident GPS-guided steps and your path is fully reproducible from the same starting fog position; set to the DDPM value, you re-introduce the random bumping and recover exactly DDPM's behavior. The bijective encoding property at σ = 0 is then natural: since every step is deterministic given the network, the reverse journey from x₀ back to xᵀ is equally well-defined, meaning you can "fog up" any real image and unfog it again exactly — a capability that enables powerful editing applications like SDEdit and prompt-to-prompt interpolation.
Historical Development
Origins · 1 minhow this was discovered
History
2015
[Sohl-Dickstein et al.] — "Deep Unsupervised Learning using Nonequilibrium Thermodynamics" introduces the foundational idea of learning to reverse a diffusion process, establishing the thermodynamic motivation for diffusion generative models.
2019
[Song & Ermon] — Score matching with Langevin dynamics (SMLD / NCSN) demonstrates that a neural network trained to predict ∇ log p(xₜ) enables sampling via annealed Langevin dynamics, prefiguring the connection between score functions and diffusion.
2020
[Ho, Jain & Abbeel (NeurIPS 2020)] — DDPM reformulates diffusion as a hierarchical VAE with a fixed encoder and demonstrates photorealistic 256×256 generation; establishes the ε-prediction parameterization and the 1000-step standard that DDIM will accelerate.
2020
[Song, Meng & Ermon (ICLR 2021, submitted Oct 2020)] — DDIM paper introduces non-Markovian forward process family, proves equivalence of ELBO objectives, derives the deterministic sampler and subsequence acceleration, and demonstrates 10–50× speedup with negligible quality loss.
2020
[Song, Sohl-Dickstein et al. (NeurIPS 2020)] — Score-based SDEs paper provides the continuous-time unification, showing DDPM as VE-SDE and demonstrating the probability flow ODE whose discretization is closely related to the DDIM sampler.
2021
[Dhariwal & Nichol (NeurIPS 2021)] — ADM/classifier guidance applies DDIM sampling as the default inference method for their state-of-the-art ImageNet diffusion model, cementing DDIM as the standard accelerated sampler.
2022
[Lu et al.] — DPM-Solver derives higher-order (2nd and 3rd order) ODE integrators for diffusion probability flow ODEs, achieving high quality in 10–20 steps by analytically computing the exponential integrator term that DDIM's first-order scheme approximates.
2022
[Rombach et al.] — Latent Diffusion Models (Stable Diffusion) operates diffusion in the compressed latent space of a VQ-VAE and uses DDIM as its primary sampler; this becomes the dominant deployed architecture for text-to-image generation.
2023
[Song et al., Consistency Models] — Consistency Models distill the DDIM ODE trajectory into a single-step model, framing consistency training explicitly as learning to map any point on a DDIM trajectory to its corresponding x₀, showing DDIM's geometric structure is central to the next generation of fast generative models.
Why It Matters
Real-World Impact
Stable Diffusion (Rombach et al., 2022)The publicly released Stable Diffusion model uses DDIM as its default sampler (--sampler ddim) and is configured to run 50 DDIM steps vs. the 1000 DDPM steps theoretically required, making interactive text-to-image generation feasible on consumer GPUs with ~4 seconds per image rather than ~80 seconds.
DALL·E 2 (OpenAI, 2022)The unCLIP architecture uses a diffusion prior and a diffusion decoder, both sampled with DDIM-based accelerated schedules; DDIM's deterministic inversion is used in the CLIP-guided editing pipeline to reconstruct source images for downstream manipulation.
DreamBooth and Textual InversionBoth personalization techniques exploit DDIM's exact inversion property — a real photograph is DDIM-inverted to a noise latent, and the optimization adjusts embeddings so that DDIM decoding of that same latent reproduces the target concept, a workflow impossible with stochastic DDPM sampling.
SDEdit (Meng et al., 2021)Image editing via stroke-guided synthesis explicitly relies on DDIM's interpolation property: a source image is forward-diffused to an intermediate timestep τ, then denoised with the DDIM sampler conditioned on a text prompt, allowing partial preservation of structure; this is one of the earliest DDIM-enabled editing paradigms.
Video Diffusion Models (Ho et al., 2022; Imagen Video)Spatiotemporal U-Net architectures generating video frames use DDIM-style sampling across all frame dimensions simultaneously; the deterministic nature of DDIM is critical for temporal consistency since independent stochastic noise injections per step would cause flicker artifacts across frames.
Consistency Models (Song et al., 2023)Consistency distillation trains a student network by enforcing self-consistency along DDIM ODE trajectories, with the DDIM trajectory providing the regression targets; this achieves single-step generation and would be ill-defined without the smooth deterministic DDIM path structure.
Medical Image Synthesis (Pinaya et al., 2022; Müller-Franzes et al., 2023)Brain MRI and histopathology synthesis pipelines adopt DDIM for latency-constrained clinical workflows; the exact reconstruction property additionally allows anomaly detection by comparing an image with its DDIM-reconstructed version, flagging regions where the model deviates from the manifold.