Generative Recursive Reasoning — part 1/5
Cleaned-text code points [0, 15740). Bibliography omitted here; source.md includes it.
# Generative Recursive Reasoning
###### Abstract
How should future neural reasoning systems implement extended computation? Recursive Reasoning Models (RRMs) offer a promising alternative to autoregressive sequence extension by performing iterative latent-state refinement with shared transition functions. Yet existing RRMs are largely deterministic, following a single latent trajectory and converging to a single prediction. We introduce Generative Recursive reAsoning Models (GRAM) , a framework that turns recursive latent reasoning into probabilistic multi-trajectory computation. GRAM models reasoning as a stochastic latent trajectory, enabling multiple hypotheses, alternative solution strategies, and inference-time scaling through both recursive depth and parallel trajectory sampling. This yields a latent-variable generative model supporting conditional reasoning via p θ ( y ∣ x ) p_{\theta}(y\mid x) and, with fixed or absent inputs, unconditional generation via p θ ( x ) p_{\theta}(x) . Trained with amortized variational inference, GRAM improves over deterministic recurrent and recursive baselines on structured reasoning and multi-solution constraint satisfaction tasks, while demonstrating an unconditional generation capability. https://ahn-ml.github.io/gram-website
## 1 Introduction
A central question for future neural reasoning systems is how extended computation should be implemented. Large autoregressive models typically scale reasoning by extending a sequence-generation process, whether intermediate computation is expressed explicitly as chain-of-thought tokens or implicitly in hidden or latent representations [ 1 , 2 , 3 , 4 , 5 , 6 ] . A complementary direction is explored by Recursive Reasoning Models (RRMs), which use repeated computation to refine a persistent latent state rather than to append new elements to an output or reasoning sequence [ 7 , 8 , 9 ] . This approach is appealing because it decouples reasoning depth from both parameter scale and output length: a compact model can perform many steps of internal computation by repeatedly applying shared transition functions over time.
Recent recursive reasoning models such as HRM [ 8 ] and TRM [ 9 ] provide early evidence for the potential of this approach in structured reasoning. Rather than producing a solution in a single feedforward pass, they perform extended computation through iterative latent-state refinement, deep supervision across refinement steps, and reasoning-oriented recurrent designs such as hierarchical latent dynamics. These features make them well suited to problems requiring constraint propagation, state tracking, iterative correction, and multi-step inference. More broadly, they build on a principle also explored in recurrent Transformer architectures such as Universal Transformers [ 10 ] and Looped Transformers [ 7 ] : shared Transformer blocks can be repeatedly applied to increase computational depth without increasing parameter count. Together, these models suggest that reasoning capability can emerge not only from scaling model size or generating longer traces, but also from the organization of computation itself.
While recurrent latent-state refinement provides an appealing mechanism for efficiently increasing reasoning depth, depth alone is not sufficient for many reasoning problems. A capable reasoning system should also be able to maintain uncertainty, consider alternative hypotheses, and explore multiple possible solution strategies [ 11 , 12 ] . This is especially important in settings where ambiguity or multiple valid solutions are intrinsic, and more generally in problems where a single refinement path may become trapped in a suboptimal reasoning trajectory. In this sense, future RRMs should be not only deep, in the sense of repeated refinement, but also wide, in the sense of maintaining and exploring multiple latent trajectories in parallel.
Existing RRMs [ 7 , 8 , 9 , 10 ] , however, remain fundamentally deterministic: given the same input and initialization, they follow a single latent trajectory and converge to a single prediction. This deterministic recursion collapses the space of plausible reasoning paths into a single attractor, leaving probabilistic multi-hypothesis latent reasoning largely unexplored within the RRM paradigm. This motivates the central question of our work: can recursive latent computation support probabilistic, generative, multi-hypothesis reasoning while preserving the efficiency of compact recurrent models?
In this paper, we propose Generative Recursive reAsoning Models (GRAM), a framework that turns recursive latent reasoning into probabilistic multi-trajectory computation. GRAM treats the reasoning process itself as a stochastic latent trajectory: at each recursion step, the model samples a transition conditioned on the input and the current reasoning state, rather than deterministically updating to a single next state. Repeating this process defines a distribution over possible reasoning trajectories, allowing the model to maintain multiple hypotheses, explore alternative solution strategies, and scale inference not only by increasing recursive depth but also by sampling trajectories in parallel. From a probabilistic perspective, GRAM is a latent-variable generative model: it models p θ ( y ∣ x ) p_{\theta}(y\mid x) by marginalizing over latent reasoning trajectories, while the same recursive process can also define an unconditional generative model p θ ( x ) p_{\theta}(x) when the input is fixed or absent.
We evaluate GRAM on controlled reasoning and generation tasks that serve as probes of the architectural properties targeted by our formulation: recursive refinement, stochastic exploration, multi-solution coverage, and inference-time scaling. Given this goal, our experiments focus on comparisons with the most relevant deterministic recurrent and recursive latent reasoning baselines, including Looped Transformers, HRM, and TRM, rather than frontier-scale general-purpose LLMs whose training data, inference budgets, and external scaffolding are not directly comparable. Sudoku-Extreme [ 8 ] and ARC-AGI [ 13 , 14 ] test structured reasoning under hard constraints and abstract transformations; N-Queens and Graph Coloring evaluate multi-solution recovery; and binarized MNIST [ 15 ] probes the unconditional generative interpretation.
Our main contribution is to establish probabilistic multi-trajectory recursion as a design principle for future recurrent and recursive reasoning architectures. Concretely, we make three contributions. First, we formulate recursive reasoning as a latent-variable generative process, where solutions are obtained by marginalizing over stochastic reasoning trajectories. Second, we introduce width-based inference-time scaling, enabling inference to scale not only with recursive depth but also with the number of sampled latent trajectories. Third, we provide empirical evidence that this formulation yields the intended architectural advantages over deterministic recurrent and recursive baselines, improving structured reasoning, multi-solution constraint satisfaction, and unconditional generation.
## 2 Generative Recursive Reasoning Models
In this section, we introduce Generative Recursive reAsoning Models (GRAM), an instantiation of probabilistic recursive reasoning. We describe the architecture in Section 2.1 and the training procedure in Section 2.2 , with an architecture schematic shown in Figure 2 .
### 2.1 Architecture
Overview. GRAM models the conditional distribution p θ ( y ∣ x ) p_{\theta}(y\mid x) by marginalizing over stochastic latent reasoning trajectories. Given an input x x , GRAM first computes an embedding e x = f enc ( x , θ ) , \displaystyle e_{x}=f_{\mathrm{enc}}(x;\theta), (1) which is reused throughout the entire recursive computation. Starting from a fixed initial latent state z 0 z_{0} , the model evolves the latent state through learned stochastic transitions. The recursive computation is organized into two nested levels: inner and outer loops.
At the inner level, a latent transition samples a new latent state conditioned on the previous latent state and the input embedding, z t ∼ p θ ( z t ∣ z t − 1 , e x ) , t = 1 , … , T . \displaystyle z_{t}\sim p_{\theta}(z_{t}\mid z_{t-1},e_{x}),\qquad t=1,\dots,T. (2) At the end of the T T transitions, the decoder produces a prediction, y ^ = arg max f dec ( z T , θ ) \hat{y}=\argmax f_{\mathrm{dec}}(z_{T};\theta) . We refer to the sequence of T T transitions from the initial state z 0 z_{0} to the final state z T z_{T} as a supervision step . A supervision step is the unit at which the decoder is invoked, and the training objective is applied, with gradients computed as described in Section 2.2 .
At the outer level, N sup N_{\mathrm{sup}} supervision steps are applied recursively, with the final state of one supervision step serving as the initial state of the next, thereby forming the full recursive computation: z 0 ( 1 ) → T transitions z T ( 1 ) = z 0 ( 2 ) → T transitions ⋯ → T transitions z T ( N sup ) , \displaystyle z_{0}^{(1)}\;\xrightarrow{\;T\text{ transitions}\;}\;z_{T}^{(1)}=z_{0}^{(2)}\;\xrightarrow{\;T\text{ transitions}\;}\;\cdots\;\xrightarrow{\;T\text{ transitions}\;}\;z_{T}^{(N_{\mathrm{sup}})}, (3) where z t ( n ) z_{t}^{(n)} denotes the latent state at the t t -th transition of the n n -th supervision step, z 0 ( 1 ) z_{0}^{(1)} is the fixed initial state, and the terminal state of one supervision step serves as the initial state of the next ( z 0 ( n + 1 ) := z T ( n ) \smash{z_{0}^{(n+1)}:=z_{T}^{(n)}} ). This abstract formulation can be instantiated with various recurrent Transformer backbones, including flat designs such as Universal Transformers and Looped Transformers [ 10 , 7 ] , as well as hierarchical designs such as HRM and TRM [ 8 , 9 ] .
Stochastic Latent Transitions. Unlike prior recursive reasoning models (RRMs) that update the latent state deterministically and follow a single fixed trajectory [ 8 , 9 ] , GRAM defines p θ ( z t ∣ z t − 1 , e x ) p_{\theta}(z_{t}\mid z_{t-1},e_{x}) as a stochastic transition, so that repeated computation induces a distribution over latent reasoning trajectories. Concretely, GRAM realizes this transition as a learned stochastic residual perturbation around a deterministic update: at each transition, the model first computes a deterministic update u t u_{t} from z t − 1 z_{t-1} and e x e_{x} , then samples a conditional perturbation from a state-dependent Gaussian, and adds it to u t u_{t} : ϵ t \displaystyle\epsilon_{t} ∼ p θ ( ϵ t ∣ u t ) := 𝒩 ( μ θ ( u t ) , σ θ 2 ( u t ) I ) , \displaystyle\sim p_{\theta}(\epsilon_{t}\mid u_{t}):=\mathcal{N}\!\left(\mu_{\theta}(u_{t}),\,\sigma_{\theta}^{2}(u_{t})I\right), (4) z t \displaystyle z_{t} = u t + ϵ t . \displaystyle=u_{t}+\epsilon_{t}. (5) We refer to ϵ t \epsilon_{t} as the learnable stochastic guidance . The mean μ θ ( u t ) \mu_{\theta}(u_{t}) encodes a state-dependent direction in which the trajectory is steered, while the variance σ θ 2 ( u t ) \sigma_{\theta}^{2}(u_{t}) controls the amount of exploration. This design allows GRAM to capture uncertainty, prevent convergence to local minima, and support robust exploration of the solution space without discarding the deterministic refinement performed by u t u_{t} .
Hierarchical Instantiation. We instantiate the latent state with two interacting components, z = ( h , l ) z=(h,l) . The high-level component h h is updated once per latent transition and carries abstract reasoning state, while the low-level component l l is updated K K times within a single transition and carries fine-grained intermediate computation. This decomposition separates the two roles across time scales, with h h accumulating slowly across transitions and l l refined rapidly within each one.
With this hierarchical multi-scale structure, a single transition z t − 1 → z t z_{t-1}\to z_{t} is computed as follows. The low-level component is first refined for K K updates, with the high-level component held fixed: l t , k = f L ( h t − 1 , l t , k − 1 , e x ; θ ) , k = 1 , … , K , \displaystyle l_{t,k}=f_{\mathrm{L}}(h_{t-1},\,l_{t,k-1},\,e_{x};\,\theta),\qquad k=1,\dots,K, (6) where l t , 0 := l t − 1 l_{t,0}:=l_{t-1} and we write l t := l t , K l_{t}:=l_{t,K} for the refined low-level component. The high-level component is then updated as a stochastic transition conditioned on the refined l t l_{t} , u t \displaystyle u_{t} = f H ( h t − 1 , l t , θ ) , \displaystyle=f_{\mathrm{H}}(h_{t-1},\,l_{t};\,\theta), (7) ϵ t \displaystyle\epsilon_{t} ∼ p θ ( ϵ t ∣ u t ) := 𝒩 ( μ θ ( u t ) , σ θ 2 ( u t ) I ) , \displaystyle\sim p_{\theta}(\epsilon_{t}\mid u_{t}):=\mathcal{N}\!\big(\mu_{\theta}(u_{t}),\,\sigma^{2}_{\theta}(u_{t})\,I\big), (8) h t \displaystyle h_{t} = u t + ϵ t , \displaystyle=u_{t}+\epsilon_{t}, (9) and we set z t = ( h t , l t ) z_{t}=(h_{t},l_{t}) . Note that stochasticity is introduced only at the high level: the low-level refinement is fully deterministic, while the stochastic guidance signal ϵ t \epsilon_{t} acts on the slower, more abstract component of the latent state, where it can steer the overall reasoning trajectory across transitions 1 1 1 We also tried injecting noise into the low-level state, but found that it did not improve performance. . Under this instantiation, the decoder reads only the high-level component, i.e., f dec ( z T ) = f dec ( h T ) f_{\mathrm{dec}}(z_{T})=f_{\mathrm{dec}}(h_{T}) . Additional architectural details are provided in Appendix B.1 .
Modeling Unconditional Distribution. While the description so far focuses on the conditional setting p θ ( y ∣ x ) p_{\theta}(y\mid x) , the same recursive process can also be defined as an unconditional generative model p θ ( x ) p_{\theta}(x) when the input is replaced with an empty conditioning embedding. We use this formulation for generation tasks in Section 4.3 .
### 2.2 Training
GRAM is trained to model the conditional distribution p θ ( y ∣ x ) p_{\theta}(y\mid x) , where each training example consists of an input x x and its corresponding target y y . As a probabilistic model, GRAM adopts a latent-variable formulation and is optimized by maximizing an evidence lower bound (ELBO) with respect to the generative parameters θ \theta and variational parameters ϕ \phi .
Latent Variable Modeling. We model GRAM as a latent-variable probabilistic model p θ p_{\theta} , where the full latent trajectory τ = ( z 0 → ⋯ → z T Total ) \tau=(z_{0}\rightarrow\cdots\rightarrow z_{T_{\mathrm{Total}}}) consists of a sequence of latent variables, with T Total = T × N sup T_{\mathrm{Total}}=T\times N_{\mathrm{sup}} . The conditional likelihood is defined as p θ ( y ∣ x ) = ∫ p θ ( y ∣ τ , x ) p θ ( τ ∣ x ) 𝑑 τ , p_{\theta}(y\mid x)=\int p_{\theta}(y\mid\tau,x)\,p_{\theta}(\tau\mid x)\,d\tau, (10) where x x denotes the input problem and y y denotes the corresponding ground-truth output.
Direct maximum likelihood estimation of log p θ ( y ∣ x ) \log p_{\theta}(y\mid x) is intractable due to the marginalization over latent trajectories. We therefore introduce a variational posterior q ϕ ( τ ∣ x , y ) q_{\phi}(\tau\mid x,y) and optimize the evidence lower bound (ELBO), jointly training θ \theta and ϕ \phi via variational inference: log p θ ( y ∣ x ) ≥ 𝔼 q ϕ ( τ ∣ x , y ) [ log p θ ( y ∣ τ , x ) ] − KL ( q ϕ ( τ ∣ x , y ) ∥ p θ ( τ ∣ x ) ) . \displaystyle\log p_{\theta}(y\mid x)\geq\mathbb{E}_{q_{\phi}(\tau\mid x,y)}[\log p_{\theta}(y\mid\tau,x)]-\mathrm{KL}\!\left(q_{\phi}(\tau\mid x,y)\,\|\,p_{\theta}(\tau\mid x)\right). (11)