Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionality

The denoising diffusion probabilistic model (DDPM) has emerged as a mainstream generative model in generative AI. While sharp convergence guarantees have been established for the DDPM, the iteration complexity is, in general, proportional to the ambient data dimension, resulting in overly conservative theory that fails to explain its practical efficiency. This has motivated the recent work Li and Yan (2024a) to investigate how the DDPM can achieve sampling speed-ups through automatic exploitation of intrinsic low dimensionality of data. We strengthen this line of work by demonstrating, in some sense, optimal adaptivity to unknown low dimensionality. For a broad class of data distributions with intrinsic dimension $k$, we prove that the iteration complexity of the DDPM scales nearly linearly with $k$, which is optimal when using KL divergence to measure distributional discrepancy. Notably, our work is closely aligned with the independent concurrent work Potaptchik et al. (2024) – posted two weeks prior to ours – in establishing nearly linear-$k$ convergence guarantees for the DDPM.

💡 Research Summary

This paper provides a significant theoretical advancement in understanding the efficiency of Denoising Diffusion Probabilistic Models (DDPMs). While sharp convergence guarantees for DDPMs exist, they typically scale with the ambient data dimension (d), failing to explain the model’s remarkable practical performance in high-dimensional settings like image generation. This gap motivates the investigation into how DDPMs can leverage the intrinsic low-dimensional structure of real-world data.

The core contribution of this work is proving that the standard DDPM sampler is optimally adaptive to unknown low-dimensional data distributions. The authors demonstrate that when the data distribution resides on or near a manifold with intrinsic dimension k (where k « d), the iteration complexity of the DDPM—the number of steps needed to generate a sample within ε^2 accuracy in KL divergence—scales nearly linearly with k. Specifically, they prove an upper bound of O(√k / ε^2) (ignoring logarithmic factors), which is optimal for this error metric.

A key technical insight is reframing the DDPM update rule as a discretization of a reparameterized Stochastic Differential Equation (SDE) using an exponential integrator scheme. In this SDE, the nonlinear component of the drift term is proportional to the posterior mean of the data given its noised state. Intuitively, when the data lies on a low-dimensional manifold, this posterior mean function acts as a “projection” onto that manifold. This inherent structure enhances the smoothness of the solution path, thereby reducing discretization error and speeding up convergence. Remarkably, the DDPM’s original update rule, derived from variational inference principles, automatically implements this adaptive discretization schedule tailored for low-dimensional data.

This result substantially improves upon prior bounds that scaled with k^4 (Li and Yan, 2024a) or k^3 (Azanguov et al., 2024), establishing near-optimal scaling. The analysis is performed under the assumption of access to perfect score functions but can be extended to account for score estimation error. The paper also notes the concurrent and independent work of Potaptchik et al. (2024), which arrived at a similar linear-in-k convergence guarantee under slightly different assumptions about the low-dimensional structure.

In summary, this research rigorously explains a key reason behind the practical efficiency of diffusion models: their inherent, optimal ability to exploit the low-dimensional geometry of data without any explicit dimensionality reduction mechanism, bridging a crucial gap between theory and practice in generative AI.