[Paper] The Geometry of Noise: Why Diffusion Models Don't Need Noise Conditioning

Source: arXiv - 2602.18428v1

Overview

A new theoretical study explains why “autonomous” diffusion models—generative networks that don’t receive an explicit noise‑level input—can still generate high‑quality samples. By interpreting the training objective as a Riemannian gradient flow on a marginal energy landscape, the authors show how a single, time‑invariant vector field implicitly learns to counteract the singular geometry that normally appears near the data manifold. This work bridges a gap between the empirical success of noise‑agnostic generators (e.g., Equilibrium Matching, blind diffusion) and a rigorous understanding of their stability.

Key Contributions

• Marginal Energy Formalism – Introduces (E_{\text{marg}}(\mathbf{u}) = -\log p(\mathbf{u})) where (p(\mathbf{u})) is the data distribution marginalised over an unknown noise level (t).
• Riemannian Gradient Flow Interpretation – Proves that autonomous diffusion sampling follows a Riemannian gradient descent on the marginal energy, not simple blind denoising.
• Geometric Singularity Cancellation – Shows that the learned time‑invariant field implicitly defines a local conformal metric that neutralises the (1/t^{p}) singularity orthogonal to the data manifold, turning an infinite potential well into a stable attractor.
• Structural Stability Conditions – Derives precise conditions under which sampling with autonomous models remains stable, providing a theoretical safety net for practitioners.
• Jensen Gap vs. Velocity Parameterizations – Identifies a “Jensen Gap” problem in noise‑prediction heads that amplifies estimation errors, while demonstrating that velocity‑based heads satisfy a bounded‑gain property and are inherently robust.

Methodology

1. Marginalisation over Noise – The authors treat the noise level (t) as a random variable with a prior (p(t)). The noisy observation density (p(\mathbf{u}|t)) is integrated to obtain the marginal density (p(\mathbf{u})).
2. Energy Decomposition – They decompose the marginal energy into a singular component (blowing up as (t \to 0)) and a regular component learned by the network.
3. Riemannian Geometry – By defining a conformal metric (g(\mathbf{u})) that scales with the learned field, the descent dynamics become a Riemannian gradient flow: (\dot{\mathbf{u}} = -g^{-1}(\mathbf{u})\nabla E_{\text{marg}}(\mathbf{u})).
4. Stability Analysis – Using tools from dynamical systems, they prove that if the metric satisfies a bounded‑gain condition, trajectories remain bounded and converge to the data manifold.
5. Parameterization Comparison – They analytically compare two common heads: (a) noise‑prediction (predicting (\epsilon)) and (b) velocity‑prediction (predicting (\mathbf{v} = -\nabla_{\mathbf{u}}E_{\text{marg}})). The former suffers from the Jensen Gap, while the latter naturally respects the bounded‑gain condition.

Results & Findings

These findings were validated on synthetic high‑dimensional manifolds and on standard image benchmarks (e.g., CIFAR‑10, LSUN), where velocity‑based autonomous models matched or exceeded the quality of traditional time‑conditioned diffusion samplers.

Practical Implications

• Simpler Model Deployment – Removing the need for a noise‑level input reduces the inference API surface, making it easier to integrate diffusion models into production pipelines (e.g., one‑call generation APIs).
• Robustness to Noise‑Schedule Mis‑specification – Since the model internally adapts to the effective noise level, developers no longer need to fine‑tune a noise schedule for each dataset or downstream task.
• Memory & Compute Savings – A single, time‑invariant network eliminates the need for multiple conditional branches or extra embeddings, shaving off a few percent of GPU memory and latency.
• Design Guidance for New Architectures – The paper recommends velocity‑based heads over noise‑prediction heads for any autonomous or “blind” diffusion variant, steering future research toward bounded‑gain parameterizations.
• Potential for Real‑Time Generation – Stability guarantees open the door to aggressive step‑size schedules (fewer diffusion steps) without sacrificing quality, which is attractive for interactive applications (e.g., image editing, video frame synthesis).

Limitations & Future Work

• Assumed Prior on Noise Levels – The analysis relies on a known prior (p(t)); mismatches between the assumed and true noise distribution could affect the implicit metric.
• High‑Dimensional Synthetic Validation – While experiments on real image datasets are encouraging, the theoretical guarantees are proven under smoothness assumptions that may not hold for all natural data manifolds.
• Extension to Conditional Generation – The current framework focuses on unconditional generation; adapting the marginal‑energy view to class‑conditional or text‑to‑image diffusion remains an open question.
• Exploration of Alternative Metrics – The conformal metric emerging from training is implicit; future work could investigate explicit metric learning to further improve stability or accelerate sampling.

Bottom line: This paper demystifies why noise‑agnostic diffusion models work, providing a solid geometric foundation and practical design rules that developers can immediately apply to build more robust, efficient generative systems.

Authors

• Mojtaba Sahraee-Ardakan
• Mauricio Delbracio
• Peyman Milanfar

Paper Information

• arXiv ID: 2602.18428v1
• Categories: cs.LG, cs.CV, eess.IV
• Published: February 20, 2026
• PDF: Download PDF