[Paper] Regularized Random Fourier Features and Finite Element Reconstruction for Operator Learning in Sobolev Space

Source: arXiv - 2512.17884v1

Overview

This paper tackles a core challenge in operator learning – the data‑driven approximation of mappings between infinite‑dimensional function spaces (e.g., the solution operator of a PDE). The authors introduce a regularized random Fourier feature (RRFF) framework, enhanced with a finite‑element reconstruction (RRFF‑FEM) step, that remains accurate even when training data are noisy and far cheaper to train than classic kernel or neural‑operator methods.

Key Contributions

• RRFF with Student‑t random features: draws frequencies from a heavy‑tailed multivariate Student’s t distribution, improving robustness to outliers and high‑frequency noise.
• Frequency‑weighted Tikhonov regularization: penalizes high‑frequency components, yielding a well‑conditioned feature matrix even with modest numbers of random features.
• Theoretical guarantees: high‑probability bounds on the extreme singular values of the random‑feature matrix; shows that using (N = O(m \log m)) features (where (m) = training samples) suffices for stable learning and provable generalization error.
• RRFF‑FEM reconstruction: after learning the operator in the random‑feature space, a finite‑element map projects the output back onto a physically meaningful function space, preserving Sobolev regularity.
• Extensive empirical validation: benchmarks on 6 PDE families (advection, Burgers’, Darcy flow, Helmholtz, Navier‑Stokes, structural mechanics) demonstrate noise‑robustness, faster training, and competitive accuracy versus state‑of‑the‑art kernel and neural operators.

Methodology

1. Random Fourier Features (RFF) – Approximate a shift‑invariant kernel (k(x,y)=\kappa(x-y)) by sampling random frequencies (\omega) and forming features (\phi_\omega(u)=\exp(i\omega^\top u)).
2. Student‑t sampling – Instead of the usual Gaussian, frequencies are drawn from a multivariate Student’s t distribution. The heavy tails give a richer set of basis functions that better capture irregularities in PDE solutions.
3. Frequency‑weighted Tikhonov regularization – The linear system (\Phi w \approx y) (where (\Phi) contains the random features of the input functions) is solved with a regularizer (\lambda | \Lambda w|_2^2). The diagonal weight matrix (\Lambda) grows with (|\omega|), damping high‑frequency components that are most susceptible to noise.
4. Finite‑Element Reconstruction – After predicting the output in the feature space, the coefficients are projected onto a finite‑element basis defined on the physical domain. This step enforces Sobolev‑space smoothness and yields a function that can be directly used in downstream simulations.
5. Theoretical analysis – By bounding the singular values of (\Phi) with matrix concentration inequalities, the authors prove that with (N = O(m\log m)) features the system is well‑conditioned with probability (1-\delta). This leads to explicit error bounds for both training and test data.

Results & Findings

• Noise robustness: RRFF‑FEM’s error grows sub‑linearly with added noise, whereas plain RFF and some neural operators degrade sharply.
• Training efficiency: Because only (N = O(m\log m)) random features are needed, the linear system solves in seconds even for (m) in the thousands, a regime where full kernel matrices become intractable.
• Accuracy: Across all tests, RRFF‑FEM stays within 1–3 % relative L2 error of the ground‑truth solution, matching or beating the best kernel/NN baselines.

Practical Implications

• Fast surrogate models: Engineers can replace expensive PDE solvers with a lightweight RRFF‑FEM surrogate that trains in minutes and evaluates in milliseconds, enabling real‑time control, optimization, or uncertainty quantification.
• Robustness to sensor noise: In fields like geophysics or fluid‑structure monitoring, data are often noisy. The frequency‑weighted regularization makes the learned operator tolerant to such imperfections without costly data‑cleaning pipelines.
• Scalable to large datasets: Since the method only needs (N \approx m\log m) features, it scales to thousands of training simulations—far beyond the cubic cost of classic kernel methods.
• Plug‑and‑play with existing FEM pipelines: The reconstruction step uses standard finite‑element bases, meaning developers can integrate RRFF‑FEM into existing FEM software (e.g., FEniCS, deal.II) with minimal code changes.
• Foundation for hybrid models: The approach can be combined with physics‑informed regularizers or embedded into multi‑fidelity frameworks, offering a pathway to more accurate, data‑driven scientific computing tools.

Limitations & Future Work

• Heavy‑tailed frequency sampling may still miss very localized features in highly heterogeneous media; adaptive sampling strategies could improve coverage.
• The current theory assumes i.i.d. training pairs and bounded noise; extending guarantees to correlated or non‑Gaussian noise remains open.
• Experiments focus on 2‑D problems; scaling to high‑dimensional (3‑D+time) domains will require careful memory management and possibly hierarchical feature constructions.
• Integration with online learning (updating the operator as new data arrive) is not addressed; future work could explore incremental RRFF updates.

Bottom line: RRFF‑FEM offers a mathematically grounded, noise‑robust, and computationally efficient route to learning PDE solution operators—making high‑fidelity surrogate modeling more accessible to developers and engineers working on real‑world simulation pipelines.

Authors

• Xinyue Yu
• Hayden Schaeffer

Paper Information

• arXiv ID: 2512.17884v1
• Categories: cs.LG, math.NA, stat.ML
• Published: December 19, 2025
• PDF: Download PDF