[Paper] Fast Gaussian Process Approximations for Autocorrelated Data

Source: arXiv - 2512.02925v1

Overview

Gaussian Processes (GPs) are a go‑to tool for flexible, non‑linear regression, but their cubic‑time scaling makes them impractical for large, temporally correlated datasets. This paper proposes a set of fast GP approximations that explicitly respect autocorrelation, avoiding the “temporal over‑fitting” that plagues naïve shortcuts. The authors demonstrate that, by blocking and decorrelating data, we can retain GP accuracy while cutting computation time dramatically.

Key Contributions

• Block‑wise decorrelation strategy: Introduces a systematic way to partition autocorrelated series into near‑independent blocks, enabling existing sparse GP approximations to be applied unchanged.
• Adaptation of three popular GP approximations (inducing‑point, structured kernel interpolation, and local GP methods) to the blocked setting, with theoretical justification.
• Comprehensive empirical evaluation on synthetic and real‑world time‑series benchmarks (climate, finance, sensor networks), showing speed‑ups of 5–30× with negligible loss in predictive accuracy.
• Open‑source implementation (Python/NumPy) that integrates with standard GP libraries (GPy, GPflow), lowering the barrier for practitioners.

Methodology

1. Identify autocorrelation length – Using standard tools (e.g., autocorrelation function, spectral density), the authors estimate a correlation horizon (L) beyond which observations are effectively independent.
2. Create blocks – The time series is sliced into overlapping windows of size ≈ (L). Within each block, a linear transformation (e.g., Cholesky of the block’s covariance) decorrelates the data, turning the block into approximately i.i.d. noise.
3. Apply existing GP approximations – After decorrelation, any fast GP method that assumes i.i.d. noise can be run on each block independently. The authors adapt three representative approximations:
   • Inducing‑point (Sparse Variational GP) – Choose inducing locations per block, solve a reduced variational objective.
   • Structured Kernel Interpolation (SKI) – Build a Kronecker‑structured grid inside each block for fast matrix‑vector products.
   • Local GP (Mixture of Experts) – Treat each block as an expert, then combine predictions via a simple weighting scheme.
4. Re‑assemble predictions – Overlapping block predictions are blended (e.g., using a tapered weighting function) to produce a smooth global forecast.

The key insight is that decorrelation makes the expensive (O(N^3)) covariance inversion unnecessary; each block can be processed in (O(m^3)) where (m \ll N).

Results & Findings

*RMSE is reported relative to the exact GP baseline.

• Accuracy: Across all experiments, the blocked approximations match the full GP within a few hundredths of a percent, confirming that decorrelation does not sacrifice predictive power.
• Scalability: The method scales linearly with the number of blocks, making it viable for streaming or online settings where new data arrive continuously.
• Robustness: Sensitivity analysis shows that modest mis‑estimation of the correlation length (L) only mildly affects performance, thanks to the overlapping block design.

Practical Implications

• Time‑series forecasting pipelines – Engineers can now embed GP models in production systems (e.g., demand forecasting, anomaly detection) without the usual cubic bottleneck.
• Edge and IoT devices – The block‑wise approach fits naturally on devices with limited memory; each block can be processed on‑device and aggregated centrally.
• Hybrid modeling – The technique can be combined with deep learning feature extractors (e.g., CNNs for sensor grids) where the GP acts as a calibrated uncertainty layer.
• Rapid prototyping – Because the method plugs into existing GP libraries, data scientists can experiment with sophisticated kernels (periodic, Matérn) while retaining speed.

Limitations & Future Work

• Block size selection relies on a good estimate of the autocorrelation horizon; highly non‑stationary series may require adaptive block sizing.
• Overlap handling adds modest computational overhead and may introduce edge artifacts if the weighting scheme is not tuned.
• Extension to multivariate (spatio‑temporal) data is not covered; the authors suggest integrating their blocking with low‑rank spatio‑temporal kernels as a next step.
• Theoretical guarantees on the error introduced by decorrelation are empirical; a formal bound would strengthen the method’s appeal for safety‑critical applications.

Overall, the paper offers a pragmatic recipe for bringing the expressive power of Gaussian Processes to the fast‑moving world of autocorrelated data, opening the door for more reliable, uncertainty‑aware models in everyday engineering workflows.

Authors

• Ahmadreza Chokhachian
• Matthias Katzfuss
• Yu Ding

Paper Information

• arXiv ID: 2512.02925v1
• Categories: cs.LG, stat.ML
• Published: December 2, 2025
• PDF: Download PDF