[Paper] Are Your Reasoning Models Reasoning or Guessing? A Mechanistic Analysis of Hierarchical Reasoning Models

Source: arXiv - 2601.10679v1

Overview

The paper investigates why hierarchical reasoning models (HRMs) – a class of neural networks that excel at puzzles like Sudoku – sometimes behave more like clever guessers than true reasoners. By dissecting the internal dynamics of HRMs, the authors uncover surprising failure modes and propose concrete tricks that turn those “guesses” into reliable solutions, pushing performance on the hardest Sudoku benchmark from 54 % to nearly 97 %.

Key Contributions

• Mechanistic diagnosis of HRMs – identifies three counter‑intuitive phenomena: (1) failure on trivially simple puzzles, (2) “grokking”‑style sudden breakthroughs during reasoning steps, and (3) existence of multiple fixed points that trap the model.
• Fixed‑point perspective – reframes HRM inference as a search for a self‑consistent solution (a fixed point) rather than a gradual logical deduction.
• Three “guess‑scaling” strategies – data augmentation, input perturbation, and model bootstrapping that increase the diversity and quality of fixed‑point guesses.
• Augmented HRM – a combined system that attains 96.9 % accuracy on the Sudoku‑Extreme benchmark, a 42‑point jump over the vanilla HRM.
• Broader insight – provides a new lens for interpreting reasoning in neural models, bridging the gap between empirical success and theoretical understanding.

Methodology

1. Fixed‑Point Formalism – The authors model each reasoning step of an HRM as an iterative function (f(\cdot)). A solution is reached when the output no longer changes (i.e., (x = f(x))).
2. Empirical Probes – They craft minimal puzzles (e.g., a Sudoku grid with a single empty cell) to test whether the fixed‑point assumption holds.
3. Step‑wise Monitoring – During inference, the model’s intermediate predictions are logged to detect abrupt correctness jumps (“grokking”).
4. Multiplicity Detection – By initializing the same puzzle with slightly different random seeds, they observe convergence to different fixed points, some of which are wrong.
5. Guess‑Scaling Techniques
   • Data Augmentation: enrich training data with transformed puzzles (rotations, digit permutations) to teach the model a richer set of fixed points.
   • Input Perturbation: add controlled noise at inference time (e.g., random masking) to force the model to explore alternative trajectories.
   • Model Bootstrapping: train multiple HRMs with different random seeds and ensemble their guesses.
6. Evaluation – All variants are benchmarked on standard Sudoku datasets, with a focus on the “Sudoku‑Extreme” split that contains the toughest puzzles.

Results & Findings

• Simple Puzzle Failure: Even a one‑cell Sudoku caused the model to diverge because the iteration never satisfied the fixed‑point condition.
• Grokking Dynamics: Accuracy stayed flat for several iterations, then jumped to 100 % in a single step, indicating a hidden phase transition in the reasoning process.
• Multiple Fixed Points: About 30 % of extreme puzzles converged to an incorrect fixed point on the first try; the scaling tricks increased the chance of hitting the correct one.

Overall, the experiments confirm that HRMs are effectively “guessing” a fixed point and that boosting the number and quality of guesses dramatically improves reliability.

Practical Implications

• Robust Puzzle Solvers: Developers building AI assistants for games, education, or constraint‑satisfaction problems can now rely on HRMs for near‑perfect Sudoku solving without massive model scaling.
• General Reasoning Pipelines: The fixed‑point view suggests that other reasoning tasks (e.g., theorem proving, program synthesis) might benefit from similar guess‑scaling techniques—augment data, perturb inputs, and ensemble models.
• Efficient Deployment: Instead of training a single gigantic model, teams can train several lightweight HRMs and combine their outputs, saving GPU memory and inference latency.
• Debugging Tools: Monitoring for “grokking” steps gives a clear signal when a model is about to succeed, enabling early‑exit strategies in latency‑sensitive applications.
• Safety & Explainability: Understanding that a model may be stuck in a wrong fixed point helps engineers design fallback checks (e.g., constraint validation) before trusting the output.

Limitations & Future Work

• Domain Specificity: The analysis focuses on Sudoku‑style constraint puzzles; it remains open how well the fixed‑point framework transfers to open‑ended reasoning (e.g., natural‑language inference).
• Scalability of Bootstrapping: Training many HRM instances incurs extra compute; future work could explore parameter‑efficient ensembles or Bayesian weight sampling.
• Theoretical Guarantees: While empirical evidence shows the benefits of guess scaling, a formal proof of convergence to the correct fixed point under augmentation is still missing.
• Adversarial Robustness: Perturbations improve guess diversity but may also expose the model to adversarial attacks; robust perturbation strategies need investigation.

The authors plan to extend their mechanistic lens to other hierarchical architectures and to formalize the relationship between fixed‑point multiplicity and model capacity.

Authors

• Zirui Ren
• Ziming Liu

Paper Information

• arXiv ID: 2601.10679v1
• Categories: cs.AI, cs.LG
• Published: January 15, 2026
• PDF: Download PDF