[Paper] Convergence of the generalization error for deep gradient flow methods for PDEs

Source: arXiv - 2512.25017v1

Overview

This paper puts deep gradient flow methods (DGFMs) for solving high‑dimensional partial differential equations (PDEs) on solid mathematical ground. By dissecting the total error into an approximation part (how well a neural net can represent the PDE solution) and a training part (how well the optimization converges), the authors prove that both vanish when the network is wide enough and training runs long enough.

Key Contributions

• Error decomposition: Formal split of the generalization error into approximation and training components for DGFMs.
• Universal approximation for PDEs: Proof that, under mild and verifiable conditions, neural networks can approximate the true PDE solution arbitrarily well as the number of neurons → ∞.
• Wide‑network gradient flow analysis: Derivation of the continuous‑time gradient flow that the training dynamics follow when the network width tends to infinity.
• Convergence guarantee: Demonstration that the training flow converges to a global minimizer as training time → ∞, implying the overall generalization error → 0.
• Bridging theory and practice: Provides a clear set of assumptions that practitioners can check for their specific PDE problems.

Methodology

1. Problem setup – The authors consider a broad class of PDEs (e.g., elliptic, parabolic) that admit a variational formulation. The PDE solution is expressed as the minimizer of a loss functional defined over a function space.
2. Neural‑net parametrization – They replace the unknown solution with a feed‑forward network (u_\theta(x)) and define a training loss that mirrors the PDE residual (often a Monte‑Carlo estimate of an integral).
3. Error split –
   • Approximation error: Distance between the true solution and the best possible network within the chosen architecture.
   • Training error: Gap between the best‑possible network and the network obtained after gradient descent.
4. Wide‑network limit – By letting the hidden layer width go to infinity, the finite‑dimensional parameter dynamics converge to a deterministic gradient flow in function space (a.k.a. mean‑field limit).
5. Asymptotic analysis – They study the long‑time behavior of this flow, showing it descends the loss functional to its global minimum under the paper’s assumptions.

The analysis stays at a level that developers can follow: think of the wide‑network limit as “the network behaves like a kernel method whose parameters evolve smoothly over time,” and the convergence proof as a guarantee that the optimizer will eventually find the exact PDE solution.

Results & Findings

• Approximation error → 0: For any ε > 0, there exists a sufficiently wide network such that the sup‑norm error between the network and the true PDE solution is < ε.
• Training error → 0: In the infinite‑width regime, the gradient flow converges to a stationary point that is a global minimizer of the loss; consequently the training error vanishes as training time → ∞.
• Overall generalization error → 0: Combining the two results, the total error of DGFMs can be made arbitrarily small by increasing network width and training duration.
• Assumption checklist: The paper lists concrete conditions (e.g., Lipschitz continuity of the PDE operator, bounded domain, existence of a unique weak solution) that are easy to verify for many engineering‑relevant PDEs.

Practical Implications

• Confidence in high‑dimensional solvers – Engineers can now rely on DGFMs for problems where traditional grid‑based methods explode (e.g., 10+ dimensions in quantitative finance, stochastic control, or molecular dynamics).
• Guidance for architecture design – The theory suggests that width matters more than depth for convergence, encouraging the use of wide, shallow networks when tackling PDEs.
• Training budget planning – Since the error shrinks with training time, practitioners can trade off between network size and compute time: a moderately wide network trained longer can achieve the same accuracy as a larger one trained briefly.
• Benchmarking and diagnostics – The error decomposition offers a diagnostic tool: if a DGFM implementation stalls, developers can check whether the bottleneck is approximation (network too small) or training (optimizer stuck).
• Integration with existing ML pipelines – Because the loss is expressed as an expectation over sampled points, DGFMs slot naturally into standard PyTorch/TensorFlow workflows, allowing the use of automatic differentiation, mini‑batching, and GPU acceleration.

Limitations & Future Work

• Infinite‑width idealization – Real networks are finite; while the theory predicts convergence as width grows, the rate at which practical widths approach the limit is not quantified.
• Training time to convergence – The proof assumes infinite training time; practical stopping criteria and the effect of stochastic optimizers (e.g., Adam) remain open questions.
• Specific PDE classes – The assumptions exclude certain non‑Lipschitz or highly irregular PDEs; extending the analysis to such cases would broaden applicability.
• Empirical validation – The paper is primarily theoretical; systematic experiments comparing predicted convergence rates with actual training curves would strengthen the bridge to industry use.

Bottom line: This work delivers the first rigorous guarantee that deep gradient flow methods can, in principle, solve high‑dimensional PDEs to any desired accuracy, giving developers a solid theoretical safety net while they push the practical limits of these powerful neural solvers.

Authors

• Chenguang Liu
• Antonis Papapantoleon
• Jasper Rou

Paper Information

• arXiv ID: 2512.25017v1
• Categories: math.NA, cs.LG, q-fin.CP, stat.ML
• Published: December 31, 2025
• PDF: Download PDF