[Paper] Learning under noisy supervision is governed by a feedback-truth gap

Source: arXiv - 2602.16829v1

Overview

A new study uncovers a universal “feedback‑truth gap” that emerges whenever a learning system (from deep nets to humans) updates its knowledge faster than it can verify the underlying task structure. In practice, this means that noisy feedback—incorrect labels, ambiguous rewards, or misleading signals—can dominate learning, leading to systematic over‑commitment to the wrong answer. The authors demonstrate the phenomenon across massive neural‑network experiments, human reversal‑learning tasks, and EEG‑recorded reward learning, showing that the gap is inevitable unless the rates of feedback assimilation and truth evaluation are perfectly matched.

Key Contributions

• Theoretical insight: Introduces a two‑timescale learning model that predicts a feedback‑truth gap whenever the feedback‑learning rate exceeds the truth‑evaluation rate, and proves the gap vanishes only when the two rates are equal.
• Large‑scale empirical validation: 2,700 training runs on 30 public datasets confirm the gap in dense neural networks; sparse‑residual architectures show a markedly reduced gap.
• Human behavioral evidence: Probabilistic reversal‑learning experiments (N = 292) reveal a transient over‑commitment to recent feedback that is later corrected, mirroring the model’s dynamics.
• Neurophysiological link: Simultaneous EEG recordings (N = 25) identify a post‑feedback neural signature that predicts the magnitude of the behavioral over‑commitment (amplification factor ≈ 10×).
• Quantitative characterization: Provides concrete effect‑size estimates (neural over‑commitment 0.04–0.10, behavioral d = 3.3–3.9) and shows how different architectures regulate the gap (memorization vs. scaffolding vs. active recovery).

Methodology

1. Two‑timescale model – The authors formalize learning as two coupled differential equations:

   • Feedback dynamics (fast) updates parameters based on the most recent label or reward.
   • Truth dynamics (slow) integrates evidence over many examples to approximate the true underlying mapping.
     The analytical solution shows a steady‑state offset (the gap) proportional to the ratio of the two rates.

2. Neural‑network experiments –

   • Datasets: 30 image/text/tabular benchmarks with synthetically injected label noise (10 %–50 %).
   • Architectures: Standard dense CNNs/MLPs, sparse‑residual networks, and a control group with matched learning rates.
   • Metrics: Gap measured as the difference between validation accuracy (truth) and training accuracy on noisy labels, tracked over 200 epochs.

3. Human reversal‑learning task – Participants choose between two options; the correct option reverses probabilistically. Feedback (correct/incorrect) is noisy. The authors compute a “commitment index” that captures how strongly a participant’s choice aligns with the most recent feedback versus the long‑term optimal policy.

4. EEG recording – While participants perform the same task, scalp EEG is recorded. A post‑feedback component (≈300 ms) is extracted and used as a non‑circular proxy for the brain’s internal “truth estimate.”

5. Statistical analysis – Mixed‑effects models assess the relationship between feedback rate, architecture sparsity, EEG amplitude, and the observed gap. Effect sizes and confidence intervals are reported throughout.

Results & Findings

Key takeaways

• The gap appears universally whenever feedback is processed faster than the truth can be inferred.
• Its size depends on the system’s ability to regulate fast feedback (architectural sparsity, cognitive control).
• In humans, a modest neural bias is magnified into a strong behavioral effect, highlighting the importance of downstream decision processes.

Practical Implications

1. Robust model design – When training on noisy data (e.g., web‑scraped labels, weak supervision), deliberately slow down the feedback loop: use smaller learning rates, gradient clipping, or incorporate truth‑estimation modules (e.g., EMA of predictions, co‑training). Sparse or residual connections can act as built‑in regulators.

2. Curriculum & self‑training – Start with high‑confidence examples (low feedback rate) and gradually introduce noisier samples, aligning the two timescales over the course of training.

3. Monitoring tools – Track the gap metric (validation vs. noisy‑train performance) in real time. A widening gap signals that the model is over‑committing to noisy feedback and may benefit from regularization or label‑cleaning.

4. Human‑in‑the‑loop systems – For crowdsourced labeling or reinforcement‑learning agents interacting with imperfect users, design interfaces that slow feedback (e.g., batch feedback, delayed rewards) to reduce over‑commitment.

5. Neuro‑inspired AI – The amplification of a small neural bias into large behavior suggests that meta‑controllers (e.g., policy‑gradient critics, attention mechanisms) could be added to AI agents to detect and correct early over‑commitment signals.

Limitations & Future Work

• Synthetic noise: Most DNN experiments used artificially corrupted labels; real‑world label noise may have structure (e.g., systematic bias) that interacts differently with the gap.
• Scope of architectures: Only dense and sparse‑residual networks were examined; transformer‑style models, graph neural nets, and recurrent nets may exhibit distinct dynamics.
• Human sample size: EEG findings are based on 25 participants; larger cohorts are needed to generalize the neural signature.
• Theoretical assumptions: The two‑timescale model assumes linear separability of feedback and truth updates; extending the theory to non‑linear, hierarchical learning processes is an open challenge.

Future research could explore adaptive learning‑rate schedules that explicitly target gap minimization, investigate the gap in continual‑learning settings where the truth evolves over time, and develop diagnostic tools that automatically suggest architectural or training‑procedure changes based on observed gap dynamics.

Authors

• Elan Schonfeld
• Elias Wisnia

Paper Information

• arXiv ID: 2602.16829v1
• Categories: cs.LG, cs.AI, cs.NE
• Published: February 18, 2026
• PDF: Download PDF