Hierarchical Reasoning Model — part 3/4

Cleaned-text code points [30231, 45585). Bibliography omitted here; source.md includes it.

The most frequently used Sudoku dataset in research, namely the Kaggle dataset 65 , can be fully solved using elementary single-digit techniques 66 . The minimal 17-clue puzzles 62 , another widely-used collection, might seem more challenging due to its small number of clues. However, this perception is misleading—since 17 represents the minimum number of clues required to guarantee a unique Sudoku solution, these hints need to be highly orthogonal to each other. This orthogonal arrangement leads to many direct, easily-resolved solution paths 67 .

We introduce Sudoku-Extreme , a more challenging dataset that is compiled from the aforementioned easy datasets as well as puzzles recognized by the Sudoku community as exceptionally difficult for human players: • Easy puzzles compiled from Kaggle, 17-clue, plus unbiased samples from the Sudoku puzzle distribution 67 : totaling 1 149 158 1\,149\,158 puzzles.

• Challenging puzzles compiled from Magictour 1465, Forum-Hard and Forum-Extreme subsets: totaling 3 104 157 3\,104\,157 puzzles.

The compiled data then undergo a strict 90/10 train-test split, ensuring that the test set puzzles cannot be derived through equivalent transformations of any training samples. Sudoku-Extreme is a down-sampled subset of this data containing 1000 training examples. We use Sudoku-Extreme in our main experiments ( Figure 1 ), which focuses on small-sample learning scenarios. To guarantee convergence and control overfitting effects in our analysis experiments ( Figures 2 , 3 and 5 ), we use the complete training data, Sudoku-Extreme-Full , containing 3 831 994 3\,831\,994 examples.

We measure puzzle difficulty by counting the number of search backtracks (“guesses”) required by a smart Sudoku solver program tdoku , which uses propositional logic to reduce the number of guesses 67 . Our Sudoku-Extreme dataset exhibits a mean difficulty of 22 22 backtracks per puzzle, significantly higher than existing datasets, including recent handmade puzzles Sudoku-Bench 68 which average just 0.45 0.45 backtracks per puzzle. These subset complexity levels are shown in Figure 6 -(d).

Maze-Hard This task involves finding the optimal path in a 30 × \times 30 maze, making it interpretable and frequently used for training LLMs in search tasks 69 , 70 , 71 . We adopt the instance generation procedure of Lehnert et al. Super , but introduce an additional filter to retain only those instances whose difficulty exceeds 110. Here, “difficulty” is defined as the length of the shortest path, which aligns with the linear time complexity of the wavefront breadth-first search algorithm on GPUs 72 . A path is considered correct if it is valid and optimal—that is, the shortest route from the start to the goal. The training and test set both include 1000 examples.

### 3.2 Evaluation Details

For all benchmarks, HRM models were initialized with random weights and trained in the sequence-to-sequence setup using the input-output pairs. The two-dimensional input and output grids were flattened and then padded to the maximum sequence length. The resulting performance is shown in Figure 1 . Remarkably, HRM attains these results with just ~1000 training examples per task—and without pretraining or CoT labels .

For ARC-AGI challenge, we start with (1) all demonstration and test input-label pairs from the training set, and (2) all demonstration pairs along with test inputs from the evaluation set. The dataset is augmented by applying translations, rotations, flips, and color permutations to the puzzles. Each task example is prepended with a learnable special token that represents the puzzle it belongs to. At test time, we proceed as follows for each test input in the evaluation set: (1) Generate and solve 1000 augmented variants and, for each, apply the inverse-augmentation transform to obtain a prediction. (2) Choose the two most popular predictions as the final outputs. 3 3 3 The ARC-AGI allows two attempts for each test input. All reported results are obtained by comparing the outputs with the withheld test labels from the evaluation set.

We augment Sudoku puzzles by applying band and digit permutations, while data augmentation is disabled for Maze tasks. Both tasks undergo only a single inference pass.

For ARC-AGI, the scores of the CoT models are taken from the official leaderboard 29 , while for Sudoku and Maze, the scores are obtained by evaluating through the corresponding API.

In Figure 1 , the baselines are grouped based on whether they are pre-trained and use CoT, or neither. The “Direct pred” baseline means using “direct prediction without CoT and pre-training”, which retains the exact training setup of HRM but swaps in a Transformer architecture. Interestingly, on ARC-AGI-1, “Direct pred” matches the performance of Liao and Gu Super , who built a carefully designed, domain-specific equivariant network for learning the ARC-AGI task from scratch, without pre-training. By substituting the Transformer architecture with HRM’s hierarchical framework and implementing ACT, we achieve more than a twofold performance improvement.

On the Sudoku-Extreme and Maze-Hard benchmarks, the performance gap between HRM and the baseline methods is significant, as the baselines almost never manage to solve the tasks. These benchmarks that demand lengthy reasoning traces are particularly difficult for CoT-based methods. With only 1000 training examples, the “Direct pred” baseline—which employs an 8-layer Transformer identical in size to HRM—fails entirely on these challenging reasoning problems. When trained on the larger Sudoku-Extreme-Full dataset, however, “Direct pred” can solve some easy Sudoku puzzles and reaches 16.9 % 16.9\% accuracy (see Figure 2 ). Lehnert et al. Super showed that a large vanilla Transformer model with 175M parameters, trained on 1 million examples across multiple trials, achieved only marginal success on 30x30 Maze tasks, with accuracy below 20 % 20\% using the p ​ a ​ s ​ s ​ @ ​ 64 pass@64 evaluation metric.

### 3.3 Visualization of intermediate timesteps

Although HRM demonstrates strong performance on complex reasoning tasks, it raises an intriguing question: what underlying reasoning algorithms does the HRM neural network actually implement? Addressing this question is important for enhancing model interpretability and developing a deeper understanding of the HRM solution space.

While a definitive answer lies beyond our current scope, we begin our investigation by analyzing state trajectories and their corresponding solution evolution. More specifically, at each timestep i i and given the low-level and high-level state pair ( z L i z_{L}^{i} and z H i z_{H}^{i} ) we perform a preliminary forward pass through the H-module to obtain z ¯ i = f H ​ ( z H i , z L i , θ H ) \bar{z}^{i}=f_{H}(z_{H}^{i},z_{L}^{i};\theta_{H}) and its corresponding decoded prediction y ¯ i = f O ​ ( z ¯ i , θ O ) \bar{y}^{i}=f_{O}(\bar{z}^{i};\theta_{O}) . The prediction y ¯ i \bar{y}^{i} is then visualized in Figure 7 .

In the Maze task, HRM appears to initially explore several potential paths simultaneously, subsequently eliminating blocked or inefficient routes, then constructing a preliminary solution outline followed by multiple refinement iterations. In Sudoku, the strategy resembles a depth-first search approach, where the model appears to explore potential solutions and backtracks when it hits dead ends. HRM uses a different approach for ARC tasks, making incremental adjustments to the board and iteratively improving it until reaching a solution. Unlike Sudoku, which involves frequent backtracking, the ARC solution path follows a more consistent progression similar to hill-climbing optimization.

Importantly, the model shows that it can adapt to different reasoning approaches, likely choosing an effective strategy for each particular task. Further research is needed to gain more comprehensive insights into these solution strategies.

## 4 Brain Correspondence

A key principle from systems neuroscience is that a brain region’s functional repertoire—its ability to handle diverse and complex tasks—is closely linked to the dimensionality of its neural representations 75 , 76 . Higher-order cortical areas, responsible for complex reasoning and decision-making, must handle a wide variety of tasks, demanding more flexible and context-dependent processing 77 . In dynamical systems, this flexibility is often realized through higher-dimensional state-space trajectories, which allow for a richer repertoire of potential computations 78 . This principle gives rise to an observable dimensionality hierarchy , where a region’s position in the processing hierarchy correlates with its effective dimensionality . To quantify this phenomenon, we can examine the Participation Ratio (PR), which serves as a standard measure of the effective dimensionality of a high-dimensional representation 79 . The PR is calculated using the formula PR = ( ∑ i λ i ) 2 ∑ i λ i 2 , \text{PR}=\frac{(\sum_{i}\lambda_{i})^{2}}{\sum_{i}\lambda_{i}^{2}}\,, where { λ i } \{\lambda_{i}\} are the eigenvalues of the covariance matrix of neural trajectories. Intuitively, a higher PR value signifies that variance is distributed more evenly across many dimensions, corresponding to a higher-dimensional representation. Conversely, a lower PR value indicates that variance is concentrated in only a few principal components, reflecting a more compact, lower-dimensional structure.

The dimensionality hierarchy can be observed, for example, in the mouse cortex, where the PR of population activity increases monotonically from low-level sensory areas to high-level associative areas, supporting this link between dimensionality and functional complexity 74 ( Figure 8 (a,b)).

We evaluated whether HRM reproduces this neuroscientific principle by calculating the PR for both recurrent modules after training on the Sudoku-Extreme Full dataset. The PR computation used the covariance matrix derived from neural states gathered across multiple Sudoku-solving trajectories. The results show a striking parallel to the biological findings. The low-level module’s state ( z L z_{L} ) occupies a relatively small subspace with a participation ratio of 30.22, whereas the high-level module’s state ( z H z_{H} ) operates in a substantially larger subspace with a participation ratio of 89.95, as shown in Figure 8 (c). Furthermore, Figure 8 (d) shows that increasing the number of unique tasks (trajectories) from 10 to 100 causes z H z_{H} dimensionality to scale up accordingly, while z L z_{L} dimensionality remains stable. These results suggest an emergent separation of representational capacity between the modules that parallels their functional roles.

To confirm that this hierarchical organization is an emergent property of training, and not an artifact of the network’s architecture, we performed a control analysis using an identical but untrained network with random weights.

We initialized an identical HRM architecture with random weights and, without any training, measured the PR of its modules as the network processed the same task-specific inputs given to the trained model.

The results, shown in Figure 8 (e,f), reveal a stark contrast: the high-level and low-level modules of the untrained network exhibit no hierarchical separation, with their PR values remaining low and nearly indistinguishable from each other. This control analysis validates that the dimensionality hierarchy is an emergent property that arises as the model learns to perform complex reasoning.

The high-to-low PR ratio in HRM ( z H / z L ≈ 2.98 z_{H}/z_{L}\approx 2.98 ) closely matches that measured in the mouse cortex ( ≈ 2.25 \approx 2.25 ). In contrast, conventional deep networks often exhibit neural collapse , where last-layer features converge to a low-dimensional subspace 80 , 81 , 82 . HRM therefore departs from the collapse pattern and instead fosters a high-dimensional representation in its higher module. This is significant because such representations are considered crucial for cognitive flexibility and are a hallmark of higher-order brain regions like the prefrontal cortex (PFC), which is central to complex reasoning.

This structural parallel suggests the model has discovered a fundamental organizational principle. By learning to partition its representations into a high-capacity, high-dimensional subspace ( z H z_{H} ) and a more specialized, low-dimensional one ( z L z_{L} ), HRM autonomously discovers an organizational principle that is thought to be fundamental for achieving robust and flexible reasoning in biological systems. This provides a potential mechanistic explanation for the model’s success on complex, long-horizon tasks that are intractable for models lacking such a differentiated internal structure. We emphasize, however, that this evidence is correlational. While a causal link could be tested via intervention (e.g., by constraining the H-module’s dimensionality), such methods are difficult to interpret in deep learning due to potential confounding effects on the training process itself. Thus, the causal necessity of this emergent hierarchy remains an important question for future investigation.

## 5 Related Work

#### Reasoning and algorithm learning

Given the central role of reasoning problems and their close relation to algorithms, researchers have long explored neural architectures that enable algorithm learning from training instances. This line of work includes Neural Turing Machines (NTM) 83 , the Differentiable Neural Computer (DNC) 84 , and Neural GPUs 85 –all of which construct iterative neural architectures that mimic computational hardware for algorithm execution, and are trained to learn algorithms from data. Another notable work in this area is Recurrent Relational Networks (RRN) 62 , which executes algorithms on graph representations through graph neural networks.

Recent studies have integrated algorithm learning approaches with Transformer-based architectures. Universal Transformers extend the standard Transformer model by introducing a recurrent loop over the layers and implementing an adaptive halting mechanism. Geiping et al. Super demonstrate that looped Transformers can generalize to a larger number of recurrent steps during inference than what they were trained on. Shen et al. Super propose adding continuous recurrent reasoning tokens to the Transformer. Finally, TransNAR 8 combine recurrent graph neural networks with language models.

Building on the success of CoT-based reasoning, a line of work have introduced fine-tuning methods that use reasoning paths from search algorithms (like A*) as SFT targets 87 , 71 , 70 .

We also mention adaptive halting mechanisms designed to allocate additional computational resources to more challenging problems. This includes the Adaptive Computation Time (ACT) for RNNs 88 and follow-up research like PonderNet 89 , which aims to improve the stability of this allocation process.

HRM further pushes the boundary of algorithm learning through a brain-inspired computational architecture that achieves exceptional data efficiency and model expressiveness, successfully discovering complex and diverse algorithms from just 1000 training examples.

#### Brain-inspired reasoning architectures