[Paper] Limitations of Learning Tanh Neural Networks with Finite Precision

Source: arXiv - 2606.11104v1

Overview

We investigate limitations of learning $\tanh$ neural networks from point evaluations under finite-precision computations and $L^p$ accuracy guarantees, building on Berner, Grohs, and Voigtländer (2023). Our approach is based on a novel construction of sharply localized bump functions via iterated $\tanh$ activations. Using this mechanism, we show that, in a finite-precision setting, no adaptive randomized algorithm based on $m$ samples can achieve a convergence rate higher than the Monte Carlo rate $O(m^{-1/p})$ in the $L^p$ norm, unless the sampling budget grows exponentially with the size of the network parameters and architecture. The results reveal fundamental limitations imposed by finite precision on the learnability of classes containing localized bump functions, extending previous results for ReLU networks to the $\tanh$ setting.

Key Contributions

This paper presents research in the following areas:

• cs.LG
• stat.ML

Methodology

Please refer to the full paper for detailed methodology.

Practical Implications

This research contributes to the advancement of cs.LG.

Authors

• Philipp Grohs
• Matěj Trödler

Paper Information

• arXiv ID: 2606.11104v1
• Categories: cs.LG, stat.ML
• Published: June 9, 2026
• PDF: Download PDF