[Paper] Symbol-Equivariant Recurrent Reasoning Models

Source: arXiv - 2603.02193v1

Overview

The paper introduces Symbol‑Equivariant Recurrent Reasoning Models (SE‑RRMs), a new family of neural architectures that can solve classic symbolic reasoning tasks—such as Sudoku and the ARC‑AGI benchmark—while explicitly respecting the inherent permutation symmetry of symbols (e.g., swapping colors or digits). By building symmetry into the model rather than relying on expensive data‑augmentation tricks, SE‑RRMs achieve state‑of‑the‑art performance with a tiny footprint (≈2 M parameters) and demonstrate impressive zero‑shot generalisation across puzzle sizes.

Key Contributions

• Symbol‑equivariant layers: Novel recurrent building blocks that guarantee identical outputs under any permutation of input symbols, eliminating the need for massive augmentation.
• Compact yet powerful architecture: SE‑RRMs match or exceed larger Recurrent Reasoning Models (RRMs) while using an order of magnitude fewer parameters.
• Cross‑size generalisation: Trained only on 9×9 Sudoku, the model solves 4×4, 16×16, and 25×25 variants without retraining—something prior RRMs could not do.
• Strong ARC‑AGI results: Competitive scores on the ARC‑AGI‑1 and ARC‑AGI‑2 tasks with minimal augmentation, showcasing applicability beyond grid‑based puzzles.
• Open‑source implementation: Full code released (https://github.com/ml-jku/SE‑RRM), facilitating reproducibility and downstream adoption.

Methodology

SE‑RRMs extend the existing Recurrent Reasoning Model family (HRM, TRM) by inserting symbol‑equivariant modules into the recurrent loop:

1. Input encoding – The puzzle grid is represented as a tensor where each cell holds a one‑hot vector for the current symbol (digit, color, etc.).
2. Equivariant transformation – A custom linear layer followed by a non‑linearity is constrained to commute with any permutation matrix applied to the symbol dimension. In practice this is achieved by sharing weights across symbols and normalising across the symbol axis.
3. Recurrent reasoning – The transformed representation is fed into a standard GRU‑style recurrent cell that iteratively refines the board state. Because the equivariant layer is applied at every step, the entire reasoning process respects symbol symmetry.
4. Output decoding – After a fixed number of iterations, a softmax over symbols yields the final board assignment.

Training uses cross‑entropy loss on the target board configuration. Crucially, no explicit data augmentation (e.g., random symbol swaps) is required—the model’s architecture already guarantees invariance.

Results & Findings

Key takeaways

• Equivariance eliminates the need for costly augmentation: Prior RRMs required thousands of symbol‑swap augmentations to reach comparable performance; SE‑RRMs achieve it out‑of‑the‑box.
• Scalable reasoning: The same parameter set generalises across board sizes, indicating that the model learns a size‑agnostic reasoning algorithm rather than memorising patterns.
• Parameter efficiency: With only 2 M weights, SE‑RRMs rival much larger transformer‑based solvers that often exceed 100 M parameters.

Practical Implications

• Lightweight AI for embedded devices: The compact architecture makes it feasible to run symbolic reasoning (e.g., puzzle generators, constraint solvers) on edge hardware such as smartphones or micro‑controllers.
• Rapid prototyping of rule‑based systems: Developers can plug SE‑RRMs into pipelines that need to enforce invariances (e.g., board‑game AI, scheduling, resource allocation) without hand‑crafting augmentation pipelines.
• Improved data efficiency: Projects with limited labelled data (common in scientific computing or custom business logic) can benefit from built‑in symmetry, reducing the annotation burden.
• Foundation for more complex symbolic AI: The equivariant design can be combined with other neural modules (e.g., graph neural networks) to handle richer relational structures while preserving symmetry guarantees.

Limitations & Future Work

• Symbol‑only equivariance: The current design handles permutations of discrete symbols but does not address spatial symmetries (rotations, reflections) that many puzzles exhibit.
• Fixed recurrence depth: The number of reasoning steps is pre‑specified; adaptive termination could improve efficiency on easier instances.
• Benchmark scope: While Sudoku and ARC‑AGI are strong proxies, broader evaluation on tasks like program synthesis, theorem proving, or real‑world constraint optimisation remains open.
• Theoretical analysis: A deeper formal study of why equivariant layers boost extrapolation across board sizes would guide further architectural refinements.

Bottom line: By weaving symmetry directly into the neural fabric, SE‑RRMs set a new standard for compact, data‑efficient symbolic reasoning—opening the door for practical AI solutions that need to respect the inherent invariances of their problem domains.

Authors

• Richard Freinschlag
• Timo Bertram
• Erich Kobler
• Andreas Mayr
• Günter Klambauer

Paper Information

• arXiv ID: 2603.02193v1
• Categories: cs.LG, cs.AI, stat.ML
• Published: March 2, 2026
• PDF: Download PDF