[Paper] Learning Dynamics Reveal a Hierarchy of Weight-Induced Layerwise Gram Metrics

Source: arXiv - 2606.09744v1

Overview

We study feed-forward ReLU networks with fixed readout and quadratic loss. The aim is to rewrite gradient descent not primarily as a dynamics in weight space, but as a collective dynamics closed in terms of fields defined on the training-set space. For a single hidden layer, the weight variables can be eliminated from the activation dynamics, yielding a closed equation for the residuals governed by a collective kernel that factorizes into an input-geometric matrix and a dynamical co-activation matrix. For deeper networks, the residual dynamics retains a clean layer-wise kernel structure. However, from depth three onward, closure requires a hierarchy of weight-induced Gram operators that mediate information transport across layers.

Key Contributions

This paper presents research in the following areas:

• cs.LG
• cond-mat.dis-nn

Methodology

Please refer to the full paper for detailed methodology.

Practical Implications

This research contributes to the advancement of cs.LG.

Authors

• Claudio Nordio

Paper Information

• arXiv ID: 2606.09744v1
• Categories: cs.LG, cond-mat.dis-nn
• Published: June 8, 2026
• PDF: Download PDF