Hierarchical Reasoning Model — part 1/4

Cleaned-text code points [0, 14504). Bibliography omitted here; source.md includes it.

# Hierarchical Reasoning Model

###### Abstract

Reasoning, the process of devising and executing complex goal-oriented action sequences, remains a critical challenge in AI. Current large language models (LLMs) primarily employ Chain-of-Thought (CoT) techniques, which suffer from brittle task decomposition, extensive data requirements, and high latency. Inspired by the hierarchical and multi-timescale processing in the human brain, we propose the Hierarchical Reasoning Model (HRM), a novel recurrent architecture that attains significant computational depth while maintaining both training stability and efficiency. HRM executes sequential reasoning tasks in a single forward pass without explicit supervision of the intermediate process, through two interdependent recurrent modules: a high-level module responsible for slow, abstract planning, and a low-level module handling rapid, detailed computations. With only 27 million parameters, HRM achieves exceptional performance on complex reasoning tasks using only 1000 training samples. The model operates without pre-training or CoT data, yet achieves nearly perfect performance on challenging tasks including complex Sudoku puzzles and optimal path finding in large mazes. Furthermore, HRM outperforms much larger models with significantly longer context windows on the Abstraction and Reasoning Corpus (ARC), a key benchmark for measuring artificial general intelligence capabilities. These results underscore HRM’s potential as a transformative advancement toward universal computation and general-purpose reasoning systems.

## 1 Introduction

Deep learning, as its name suggests, emerged from the idea of stacking more layers to achieve increased representation power and improved performance 1 , 2 . However, despite the remarkable success of large language models, their core architecture is paradoxically shallow 3 . This imposes a fundamental constraint on their most sought-after capability: reasoning. The fixed depth of standard Transformers places them in computational complexity classes such as A ​ C 0 AC^{0} or T ​ C 0 TC^{0} 4 , preventing them from solving problems that require polynomial time 5 , 6 . LLMs are not Turing-complete and thus they cannot, at least in a purely end-to-end manner, execute complex algorithmic reasoning that is necessary for deliberate planning or symbolic manipulation tasks 7 , 8 . For example, our results on the Sudoku task show that increasing Transformer model depth can improve performance, 1 1 1 Simply increasing the model width does not improve performance here. but performance remains far from optimal even with very deep models (see Figure 2 ), which supports the conjectured limitations of the LLM scaling paradigm 9 .

The LLMs literature has relied largely on Chain-of-Thought (CoT) prompting for reasoning 10 . CoT externalizes reasoning into token-level language by breaking down complex tasks into simpler intermediate steps, sequentially generating text using a shallow model 11 . However, CoT for reasoning is a crutch, not a satisfactory solution. It relies on brittle, human-defined decompositions where a single misstep or a misorder of the steps can derail the reasoning process entirely 12 , 13 . This dependency on explicit linguistic steps tethers reasoning to patterns at the token level. As a result, CoT reasoning often requires significant amount of training data and generates a large number of tokens for complex reasoning tasks, resulting in slow response times. A more efficient approach is needed to minimize these data requirements 14 .

Towards this goal, we explore “latent reasoning”, where the model conducts computations within its internal hidden state space 15 , 16 . This aligns with the understanding that language is a tool for human communication, not the substrate of thought itself 17 ; the brain sustains lengthy, coherent chains of reasoning with remarkable efficiency in a latent space, without constant translation back to language. However, the power of latent reasoning is still fundamentally constrained by a model’s effective computational depth . Naively stacking layers is notoriously difficult due to vanishing gradients, which plague training stability and effectiveness 1 , 18 . Recurrent architectures, a natural alternative for sequential tasks, often suffer from early convergence, rendering subsequent computational steps inert, and rely on the biologically implausible, computationally expensive and memory intensive Backpropagation Through Time (BPTT) for training 19 .

The human brain provides a compelling blueprint for achieving the effective computational depth that contemporary artificial models lack. It organizes computation hierarchically across cortical regions operating at different timescales, enabling deep, multi-stage reasoning 20 , 21 , 22 . Recurrent feedback loops iteratively refine internal representations, allowing slow, higher-level areas to guide, and fast, lower-level circuits to execute—subordinate processing while preserving global coherence 23 , 24 , 25 . Notably, the brain achieves such depth without incurring the prohibitive credit-assignment costs that typically hamper recurrent networks from backpropagation through time 19 , 26 .

Inspired by this hierarchical and multi-timescale biological architecture, we propose the Hierarchical Reasoning Model (HRM). HRM is designed to significantly increase the effective computational depth. It features two coupled recurrent modules: a high-level (H) module for abstract, deliberate reasoning, and a low-level (L) module for fast, detailed computations. This structure avoids the rapid convergence of standard recurrent models through a process we term “hierarchical convergence.” The slow-updating H-module advances only after the fast-updating L-module has completed multiple computational steps and reached a local equilibrium, at which point the L-module is reset to begin a new computational phase.

Furthermore, we propose a one-step gradient approximation for training HRM, which offers improved efficiency and eliminates the requirement for BPTT. This design maintains a constant memory footprint ( O ⁡ ( 1 ) O(1) compared to BPTT’s O ⁡ ( T ) O(T) for T T timesteps) throughout the backpropagation process, making it scalable and more biologically plausible.

Leveraging its enhanced effective depth, HRM excels at tasks that demand extensive search and backtracking. Using only 1,000 input-output examples, without pre-training or CoT supervision , HRM learns to solve problems that are intractable for even the most advanced LLMs. For example, it achieves near-perfect accuracy in complex Sudoku puzzles ( Sudoku-Extreme Full ) and optimal pathfinding in 30x30 mazes, where state-of-the-art CoT methods completely fail (0% accuracy). In the Abstraction and Reasoning Corpus (ARC) AGI Challenge 27 , 28 , 29 - a benchmark of inductive reasoning - HRM, trained from scratch with only the official dataset (~1000 examples), with only 27M parameters and a 30x30 grid context (900 tokens), achieves a performance of 40.3% , which substantially surpasses leading CoT-based models like o3-mini-high (34.5%) and Claude 3.7 8K context (21.2%), despite their considerably larger parameter sizes and context lengths, as shown in Figure 1 . This represents a promising direction toward the development of next-generation AI reasoning systems with universal computational capabilities.

## 2 Hierarchical Reasoning Model

We present the HRM, inspired by three fundamental principles of neural computation observed in the brain:

• Hierarchical processing: The brain processes information across a hierarchy of cortical areas. Higher-level areas integrate information over longer timescales and form abstract representations, while lower-level areas handle more immediate, detailed sensory and motor processing 20 , 22 , 21 .

• Temporal Separation: These hierarchical levels in the brain operate at distinct intrinsic timescales, reflected in neural rhythms (e.g., slow theta waves, 4–8 Hz and fast gamma waves, 30–100 Hz) 30 , 31 . This separation allows for stable, high-level guidance of rapid, low-level computations 32 , 33 .

• Recurrent Connectivity: The brain features extensive recurrent connections. These feedback loops enable iterative refinement, yielding more accurate and context-sensitive representations at the cost of additional processing time. Additionally, the brain largely avoids the problematic deep credit assignment problem associated with BPTT 19 .

The HRM model consists of four learnable components: an input network f I ​ ( ⋅ , θ I ) f_{I}(\cdot;\theta_{I}) , a low-level recurrent module f L ​ ( ⋅ , θ L ) f_{L}(\cdot;\theta_{L}) , a high-level recurrent module f H ​ ( ⋅ , θ H ) f_{H}(\cdot;\theta_{H}) , and an output network f O ​ ( ⋅ , θ O ) f_{O}(\cdot;\theta_{O}) . The model’s dynamics unfold over N N high-level cycles of T T low-level timesteps each 2 2 2 While inspired by temporal separation in the brain, our model’s “high-level” and “low-level” modules are conceptual abstractions and do not map directly to specific neural oscillation frequencies. . We index the total timesteps of one forward pass by i = 1 , … , N × T i=1,\dots,N\times T . The modules f L f_{L} and f H f_{H} each keep a hidden state— z L i z_{L}^{i} for f L f_{L} and z H i z_{H}^{i} for f H f_{H} —which are initialized with the vectors z L 0 z_{L}^{0} and z H 0 z_{H}^{0} , respectively.

The HRM maps an input vector x x to an output prediction vector y ^ \hat{y} as follows. First, the input x x is projected into a working representation x ~ \tilde{x} by the input network: x ~ = f I ​ ( x , θ I ) . \tilde{x}=f_{I}(x;\theta_{I})\;. At each timestep i i , the L-module updates its state conditioned on its own previous state, the H-module’s current state (which remains fixed throughout the cycle), and the input representation. The H-module only updates once per cycle (i.e., every T T timesteps) using the L-module’s final state at the end of that cycle: z L i \displaystyle z_{L}^{i} = f L ​ ( z L i − 1 , z H i − 1 , x ~ , θ L ) , \displaystyle=f_{L}\left(z_{L}^{i-1},z_{H}^{i-1},\tilde{x};\theta_{L}\right)\,, z H i \displaystyle z_{H}^{i} = { f H ​ ( z H i − 1 , z L i − 1 , θ H ) if ​ i ≡ 0 ​ ( mod ​ T ) , z H i − 1 otherwise . \displaystyle=\begin{cases}f_{H}\left(z_{H}^{i-1},z_{L}^{i-1};\theta_{H}\right)&\text{if }i\equiv 0\ (\mathrm{mod}\ T)\,,\\ z_{H}^{i-1}&\text{otherwise }\;.\end{cases}

Finally, after N N full cycles, a prediction y ^ \hat{y} is extracted from the hidden state of the H-module:

y ^ = f O ​ ( z H N ​ T , θ O ) . \hat{y}=f_{O}(z_{H}^{NT};\theta_{O})\;.

This entire N ​ T NT -timestep process represents a single forward pass of the HRM. A halting mechanism (detailed later in this section) determines whether the model should terminate, in which case y ^ \hat{y} will be used as the final prediction, or continue with an additional forward pass.

#### Hierarchical convergence

Although convergence is crucial for recurrent networks, standard RNNs are fundamentally limited by their tendency to converge too early. As the hidden state settles toward a fixed point, update magnitudes shrink, effectively stalling subsequent computation and capping the network’s effective depth. To preserve computational power, we actually want convergence to proceed very slowly–but engineering that gradual approach is difficult, since pushing convergence too far edges the system toward instability.

HRM is explicitly designed to counteract this premature convergence through a process we term hierarchical convergence . During each cycle, the L-module (an RNN) exhibits stable convergence to a local equilibrium . This equilibrium, however, depends on the high-level state z H z_{H} supplied during that cycle. After completing the T T steps, the H-module incorporates the sub-computation’s outcome (the final state z L z_{L} ) and performs its own update. This z H z_{H} update establishes a fresh context for the L-module, essentially “restarting” its computational path and initiating a new convergence phase toward a different local equilibrium.

This process allows the HRM to perform a sequence of distinct, stable, nested computations, where the H-module directs the overall problem-solving strategy and the L-module executes the intensive search or refinement required for each step. Although a standard RNN may approach convergence within T T iterations, the hierarchical convergence benefits from an enhanced effective depth of N ​ T NT steps. As empirically shown in Figure 3 , this mechanism allows HRM both to maintain high computational activity (forward residual) over many steps (in contrast to a standard RNN, whose activity rapidly decays) and to enjoy stable convergence. This translates into better performance at any computation depth, as illustrated in Figure 2 .

#### Approximate gradient

Recurrent models typically use BPTT to compute gradients. However, BPTT requires storing the hidden states from the forward pass and then combining them with gradients during the backward pass, which demands O ⁡ ( T ) O(T) memory for T timesteps. This heavy memory burden forces smaller batch sizes and leads to poor GPU utilization, especially for large-scale networks. Additionally, because retaining the full history trace through time is biologically implausible, it is unlikely that the brain implements BPTT 19 .

Fortunately, if a recurrent neural network converges to a fixed point, we can avoid unrolling its state sequence by applying backpropagation in a single step at that equilibrium point. Moreover, such a mechanism could plausibly be implemented in the brain using only local learning rules 34 , 35 . Based on this finding, we propose a one-step approximation of the HRM gradient–using the gradient of the last state of each module and treating other states as constant. The gradient path is, therefore, Output head → final state of the H-module → final state of the L-module → input embedding

The above method needs O ⁡ ( 1 ) O(1) memory, does not require unrolling through time, and can be easily implemented with an autograd framework such as PyTorch, as shown in Figure 4 . Given that each module only needs to back-propagate errors through its most recent local synaptic activity, this approach aligns well with the perspective that cortical credit assignment relies on short-range, temporally local mechanisms rather than on a global replay of activity patterns.