[Paper] A Backpropagation-Free Feedback-Hebbian Network for Continual Learning Dynamics

Source: arXiv - 2601.06758v1

Overview

This paper explores whether a tiny neural network that doesn’t use back‑propagation can still exhibit the kinds of learning dynamics needed for continual learning—i.e., acquiring new tasks without catastrophically forgetting old ones. By wiring a simple feed‑forward predictor together with a dedicated feedback pathway that re‑creates past activity, the author shows that purely local Hebbian updates can produce interpretable patterns of retention and forgetting on a controlled association task.

Key Contributions

• Back‑propagation‑free architecture: Introduces a compact prediction‑reconstruction network with two feed‑forward and two feedback layers, all trained by a single local learning rule.
• Unified Hebbian‑plus‑supervised rule: Combines centered Hebbian covariance, Oja‑style weight stabilization, and a local supervised drive (when target labels are present) without any weight transport.
• Continual‑learning dynamics: Demonstrates LTD‑like suppression of earlier associations and preservation of a “trace” in feedback weights when training sequentially (A → B).
• Interleaved training behavior: Shows that deterministic interleaving (A, B, A, B, …) allows both associations to coexist, rather than one overwriting the other.
• Ablation analysis: Isolates the contributions of the feedback pathway and each term of the learning rule, clarifying why regeneration and co‑maintenance emerge.
• Transparent mechanistic insight: Provides layer‑wise activity snapshots and weight‑trajectory visualizations that make the learning process easy to interpret.

Methodology

1. Network layout

   • Feed‑forward stack: Input → hidden → output (two layers) that learns a direct input‑to‑label mapping.
   • Feedback stack: Two layers that take the current hidden activity, reconstruct the previous hidden state, and add it back as an additive temporal context to the hidden layer.

2. Training schedule

   • Sequential: Train on association A until convergence, then switch to association B.
   • Deterministic interleaving: Alternate single examples of A and B throughout training.

3. Learning rule (per synapse)

   [ \Delta w = \eta\big[,\underbrace{(x-\bar{x})(y-\bar{y})}_{\text{centered Hebbian}}

   • \underbrace{\lambda w}_{\text{Oja stabilization}}
   • \underbrace{\gamma,\text{target_drive}}_{\text{local supervised term}},\big] ]
   • No global error signals, no gradient back‑propagation, and no separate forward/backward weight copies.

4. Metrics

   • Connectivity snapshots: Row/column means of each weight matrix over training phases.
   • Retention index: Normalized measure of how much the original association is preserved after learning the second one.
   • Activity visualizations: Hidden‑layer responses to each input across epochs.

Results & Findings

The visualizations show that the feedback pathway re‑injects a softened version of the earlier hidden representation, which stabilizes the corresponding output synapses even when new data arrives. The Oja term prevents runaway weight growth, keeping the system numerically stable.

Practical Implications

• Edge AI & Tiny Devices: The entire learning pipeline runs with local, per‑synapse updates, making it suitable for hardware where memory bandwidth and energy are at a premium (e.g., neuromorphic chips, micro‑controllers).
• Continual‑learning without Replay: By preserving a latent trace in feedback connections, the network sidesteps the need for costly experience replay buffers, a common bottleneck in on‑device learning.
• Interpretability: Because each weight update follows a transparent rule, developers can reason about why forgetting occurs and intervene (e.g., by adjusting the Oja decay).
• Hybrid Systems: The architecture can be stacked on top of conventional deep nets, providing a local plasticity module that handles rapid adaptation while the bulk of the network remains trained offline with back‑propagation.
• Neuroscience‑inspired AI: Demonstrates that biologically plausible learning (Hebbian + feedback) can achieve functional continual‑learning behavior, encouraging more research into brain‑inspired mechanisms for AI robustness.

Limitations & Future Work

• Scale: Experiments are limited to a tiny two‑pair association task; it remains unclear how the approach scales to high‑dimensional data (e.g., images, language).
• Task Diversity: Only supervised classification with static targets is examined; continual reinforcement learning or unsupervised settings are not addressed.
• Feedback Capacity: The feedback pathway is a simple linear reconstruction; richer recurrent or attention‑based feedback could improve trace fidelity but adds complexity.
• Theoretical Guarantees: While empirical retention indices are reported, formal bounds on forgetting or stability are not provided.
• Hardware Validation: No implementation on actual neuromorphic hardware is shown; future work could benchmark energy and latency benefits.

Overall, the paper offers a compelling proof‑of‑concept that local Hebbian learning with dedicated feedback can support continual‑learning dynamics, opening a path toward lightweight, interpretable adaptive AI systems.

Authors

• Josh Li

Paper Information

• arXiv ID: 2601.06758v1
• Categories: cs.NE, cs.LG
• Published: January 11, 2026
• PDF: Download PDF