Generative Recursive Reasoning — part 2/5

Cleaned-text code points [15740, 31046). Bibliography omitted here; source.md includes it.

During training, latent trajectories are sampled from the variational posterior q ϕ ( ⋅ ∣ x , y ) q_{\phi}(\cdot\mid x,y) , which has access to both the input problem x x and the target output y y . At inference time, where y y is unavailable, trajectories are instead generated from the learned prior p θ ( ⋅ ∣ x ) p_{\theta}(\cdot\mid x) .

Both the prior and the posterior are modeled as conditional Markov processes over latent states: p θ ​ ( τ ∣ x ) = p ⁡ ( z 0 ) ​ ∏ t = 1 T Total p θ ​ ( z t ∣ z t − 1 , x ) , q ϕ ​ ( τ ∣ x , y ) = p ⁡ ( z 0 ) ​ ∏ t = 1 T Total q ϕ ​ ( z t ∣ z t − 1 , x , y ) . p_{\theta}(\tau\mid x)=p(z_{0})\prod_{t=1}^{T_{\mathrm{Total}}}p_{\theta}(z_{t}\mid z_{t-1},x),\qquad q_{\phi}(\tau\mid x,y)=p(z_{0})\prod_{t=1}^{T_{\mathrm{Total}}}q_{\phi}(z_{t}\mid z_{t-1},x,y). (12) Here, z 0 z_{0} is a fixed initial state shared by the prior and posterior. Both transitions are implemented by adding reparameterized Gaussian noise ϵ t \epsilon_{t} after a deterministic update u t u_{t} ; the posterior uses the same transition module as the prior, but samples from a target-conditioned noise distribution q ϕ ​ ( ϵ t ∣ u t , y ) q_{\phi}(\epsilon_{t}\mid u_{t},y) , whereas the prior uses p θ ​ ( ϵ t ∣ u t ) p_{\theta}(\epsilon_{t}\mid u_{t}) .

Since the two processes share the same Markov structure and all stochasticity is introduced through ϵ 1 : T Total \epsilon_{1:T_{\mathrm{Total}}} , their trajectory distributions can be equivalently represented in noise space. Moreover, since GRAM decodes the output only from the terminal latent state, the likelihood term satisfies p θ ​ ( y ∣ τ , x ) = p θ ​ ( y ∣ z T Total , x ) p_{\theta}(y\mid\tau,x)=p_{\theta}(y\mid z_{T_{\mathrm{Total}}},x) . Therefore, the full trajectory-level ELBO can be written as ℒ ELBO = 𝔼 q ϕ [ log p θ ( y ∣ z T Total , x ) ] − ∑ t = 1 T Total 𝔼 q ϕ ​ ( ϵ < t ∣ x , y ) [ KL ( q ϕ ( ϵ t ∣ u t , y ) ∥ p θ ( ϵ t ∣ u t ) ) ] . \displaystyle\mathcal{L}_{\mathrm{ELBO}}=\mathbb{E}_{q_{\phi}}\big[\log p_{\theta}(y\mid z_{T_{\mathrm{Total}}},x)\big]-\sum_{t=1}^{T_{\mathrm{Total}}}\mathbb{E}_{q_{\phi}(\epsilon_{<t}\mid x,y)}\Big[\mathrm{KL}\!\left(q_{\phi}(\epsilon_{t}\mid u_{t},y)\,\|\,p_{\theta}(\epsilon_{t}\mid u_{t})\right)\Big]. (13) Here, u t = f H ​ ( h t − 1 , l t ) u_{t}=f_{\mathrm{H}}(h_{t-1},l_{t}) denotes the deterministic high-level update before noise injection, as defined in Equation 9 . Since u t u_{t} depends on h t − 1 h_{t-1} , which is determined by the previously sampled noise variables ϵ < t := ( ϵ 1 , … , ϵ t − 1 ) \epsilon_{<t}:=(\epsilon_{1},\dots,\epsilon_{t-1}) , the expectation averages over these ancestral samples.

Practical Implementation. In practice, following previous recursive reasoning models [ 8 , 9 ] , we train GRAM with deep supervision over N sup N_{\mathrm{sup}} consecutive supervision steps, each consisting of T T recursive latent transitions. This provides dense learning signals along the full latent trajectory, rather than supervising only the final state after T Total = T × N sup T_{\mathrm{Total}}=T\times N_{\mathrm{sup}} transitions. The terminal state of each step is reused as the initial state of the next step.

Following standard practice for recurrent models with long computation chains, we apply truncated gradient propagation [ 16 , 17 ] , as used in recent recursive reasoning models [ 8 , 9 , 18 ] . In our implementation, gradients are propagated only through the final transition of each supervision step, z T − 1 ( n ) → z T ( n ) \smash{z_{T-1}^{(n)}\to z_{T}^{(n)}} . This gives the following surrogate objective for each supervision step: ℒ GRAM ( n ) ( x , y ; θ , ϕ ) = 𝔼 q ϕ [ log p θ ( y ∣ z T ( n ) , x ) ] − KL ( q ϕ ( ϵ T ( n ) ∣ u T ( n ) , y ) ∥ p θ ( ϵ T ( n ) ∣ u T ( n ) ) ) , \displaystyle{\mathcal{L}}_{\mathrm{GRAM}}^{(n)}(x,y;\theta,\phi)=\mathbb{E}_{q_{\phi}}\big[\log p_{\theta}(y\mid z_{T}^{(n)},x)\big]-\mathrm{KL}\big(q_{\phi}(\epsilon_{T}^{(n)}\mid u_{T}^{(n)},y)\,\|\,p_{\theta}(\epsilon_{T}^{(n)}\mid u_{T}^{(n)})\big), (14) where z T ( n ) z_{T}^{(n)} is the terminal state of the current supervision step n n , and gradients are stopped through preceding states. Thus, ℒ GRAM \mathcal{L}_{\mathrm{GRAM}} should be viewed as a truncated surrogate objective rather than the exact ELBO; it introduces a biased but memory-efficient approximation to the full ELBO. Further analysis of this approximation is provided in Appendix A.3 , and detailed training hyperparameters are listed in Appendix B.2 .

### 2.3 Inference-Time Scaling

GRAM supports two complementary axes of inference-time scaling: depth , by varying the number of recursive transitions, and width , by sampling multiple latent reasoning trajectories in parallel. For depth, we follow prior recursive reasoning models [ 8 , 9 ] in adopting adaptive computation time (ACT) [ 8 , 9 , 10 ] , which allows each trajectory to terminate at a learned halting depth (details in Appendix A.1 ). For width — the focus of this section — we draw { τ ( i ) } i = 1 N ∼ p θ ​ ( τ ∣ x ) \smash{\{\tau^{(i)}\}_{i=1}^{N}\sim p_{\theta}(\tau\mid x)} from the learned prior and decode each terminal state into a candidate output y ^ ( i ) = f dec ​ ( z T ( i ) ) \smash{\hat{y}^{(i)}=f_{\mathrm{dec}}(z_{T}^{(i)})} , exploring multiple stochastic reasoning paths simultaneously rather than extending a single trajectory.

To select among candidates, we use either majority voting or best-of-N with a Latent Process Reward Model (LPRM). The LPRM is a value head v ψ ​ ( z t ) v_{\psi}(z_{t}) trained to predict the final quality of a trajectory from its latent state, using a regression target r ∈ [ 0 , 1 ] r\in[0,1] given by the final prediction accuracy. At inference time, majority voting selects the most frequent prediction, whereas LPRM-guided selection chooses the candidate with the highest predicted terminal value. Details of LPRM training are provided in Appendix A.2 . Overall, this procedure improves robustness and solution quality through parallel exploration, without increasing the sequential recursion length.

## 3 Related Work

Latent Reasoning. Latent reasoning aims to reduce the inefficiency and verbosity of explicit Chain-of-Thought (CoT) by shifting part or all of the reasoning process into latent or continuous representations [ 1 , 2 , 3 , 4 , 5 , 6 ] . By avoiding token-by-token generation of intermediate steps, such representations can make reasoning traces more compact and reduce generation overhead. Existing approaches instantiate this idea through hidden states, latent or soft tokens, continuous thoughts, internal reasoning traces, and recursive state updates for scaling test-time computation [ 4 , 7 , 19 , 20 , 21 , 22 , 23 , 18 , 24 , 25 , 26 ] . However, many remain organized around autoregressive sequence generation, where additional computation is tied to generating more tokens, latent positions, or sequential reasoning states.

Recursive Architectures. Recursive architectures perform iterative state updates and have evolved from RNNs to weight-sharing Transformers with adaptive computation [ 7 , 10 , 27 , 28 , 29 , 30 , 31 , 32 , 25 ] . Recent recursive reasoning models show that increasing inference-time depth can outperform larger static models [ 8 , 9 , 18 , 24 ] . GRAM builds on this line but formulates recurrence as a probabilistic process: instead of following a single deterministic refinement path, it maintains stochastic latent trajectories, enabling multi-path exploration and generative sampling.

Probabilistic Latent State-Space Models. Probabilistic recurrent models use stochastic latent transitions to capture uncertainty and multimodal dynamics, often trained with variational inference [ 33 , 34 , 35 , 36 , 37 , 38 ] . They have been widely used in sequential generative modeling, video prediction, and model-based reinforcement learning. GRAM shares this latent state-space view but reinterprets stochastic dynamics as computation rather than temporal observation modeling: latent transitions define possible reasoning trajectories, supporting multi-hypothesis exploration and both conditional p θ ​ ( y ∣ x ) p_{\theta}(y\mid x) and unconditional p θ ​ ( x ) p_{\theta}(x) generation.

## 4 Experiments

GRAM is designed as an architecture for probabilistic recursive reasoning, rather than as a general-purpose large language reasoning model whose training data, inference budgets, prompting strategies, tool use, and external scaffolding are not directly comparable. Following prior work on recurrent and recursive reasoning models [ 8 , 9 ] , we therefore evaluate GRAM on standard structured reasoning tasks that probe the computational properties targeted by our formulation: iterative latent refinement, stochastic trajectory exploration, multi-solution coverage, and inference-time scaling.

In the following, we first evaluate structured reasoning performance on Sudoku-Extreme and ARC-AGI (Section 4.1 ). We then assess multi-solution behavior on N-Queens and Graph Coloring (Section 4.2 ). Next, we examine the unconditional generative interpretation of GRAM on binarized MNIST (Section 4.3 ). Finally, we perform ablation studies to evaluate the impact of key design choices (Section 4.4 ).

### 4.1 Challenging Puzzle Tasks

Setup. We evaluate on Sudoku-Extreme [ 8 ] , which contains 9 × \times 9 puzzles with minimal clues requiring extensive constraint propagation, and ARC-AGI Challenge [ 13 , 14 ] , which tests abstract visual reasoning through few-shot pattern recognition. We compare against direct prediction (Transformer [ 39 ] ), a flat recursive baselines (Looped TF [ 7 ] , HRM [ 8 ] , TRM [ 9 ] ). Reported large reasoning model results [ 40 ] are included as external reference points for benchmark difficulty, rather than as controlled baselines, since their training and inference settings are not directly comparable to task-specific recursive models. For the scaling analysis, all baselines (Looped TF, HRM, TRM) are reproduced under identical settings following Yang et al. [7] and Jolicoeur-Martineau [9] .

Stochastic Guidance Improves Reasoning. Figure 3 and Table 8 summarize our main results. GRAM consistently outperforms prior recursive models across all benchmarks. We attribute this improvement to the fundamental difference in how reasoning trajectories are utilized. While Looped TF, HRM, and TRM are restricted to learning from a single deterministic path, GRAM leverages stochastic transitions to explore diverse reasoning trajectories. By training on this richer distribution of solution paths, GRAM acquires more robust reasoning capabilities, allowing it to navigate complex problem spaces more effectively than models constrained to a single sequential refinement process. Detailed experiment results, including more state-of-art methods, are provided in Appendix D.1 .

Parallel Sampling Provides a New Test-time Scaling Axis. Figure 4 (left) shows that increasing the number of parallel samples consistently improves performance across all iteration counts. Notably, GRAM with N = 20 N=20 samples at 16 iterations outperforms all deterministic baselines at 320 iterations, including TRM (97.0% vs 90.5%), despite comparable computational budget. While deterministic recursive models scale only through sequential refinement, GRAM leverages stochastic transitions to explore multiple reasoning paths in parallel. To select the best trajectory, we employ a Latent Process Reward Model (LPRM) that predicts output correctness (Section 2.3 ). This parallel scaling bypasses the latency bottlenecks of depth-based scaling while achieving superior performance. Additional analysis on the ARC-AGI Challenge is provided in Appendix D.2 .

### 4.2 Multi-solution Puzzle Tasks

Setup. To evaluate whether GRAM can capture diverse solutions, we test on N-Queens ( 8 × 8 8\times 8 , 10 × 10 10\times 10 ) and Graph Coloring (8-vertex, 10-vertex) tasks, where multiple valid solutions exist for each input. We compare against direct prediction (Transformer [ 39 ] ), recursive models (Looped TF [ 7 ] , HRM [ 8 ] , TRM [ 9 ] ), and generative models (Autoregressive Transformer (AR), MDLM [ 41 ] ). For N-Queens, we report accuracy (whether the output satisfies all constraints) and coverage (found / total valid solutions, with 20 samples). For Graph Coloring, we report conflict edges (number of constraint violations; lower is better) instead of accuracy. Detailed configurations are provided in Appendix C.2 .

Deterministic Recursion Fails on Multi-Solution Tasks. Table 1 reveals that deterministic recursive models structurally cannot capture multiple solutions, with coverage at most 36.1% across all tasks. Figure 4 (right) further illustrates this limitation: as the number of valid solutions increases, all three deterministic recursive baselines exhibit sharp accuracy degradation, whereas GRAM maintains consistent performance regardless of solution count. This confirms that deterministic latent updates cause mode collapse when multiple valid outputs exist for the same input. Additional coverage analysis is provided in Appendix D.3 .

Recursive Refinement Yields Sharper Constraint Satisfaction. While generative models (AR, MDLM) achieve high coverage, GRAM consistently attains higher accuracy with comparable diversity. On N-Queens, GRAM reaches 99.7% accuracy versus 96.3% (AR) and 96.1% (MDLM). The gap is more pronounced on Graph Coloring, where GRAM reduces conflict edges to 2.7 and 3.3 on 8- and 10-vertex tasks, compared to 19.0 and 61.3 for AR. This demonstrates that recursive refinement enables stricter constraint satisfaction than generative sampling alone.

N-Queens Graph Coloring 8 × 8 8\times 8 10 × 10 10\times 10 8-vertex 10-vertex Method Rec. Gen. # Params Accuracy Coverage Accuracy Coverage Conflict ↓ \downarrow Coverage Conflict ↓ \downarrow Coverage Direct Pred (8 layers) ✗ ✗ 27M 40.4 ± 1.1 \pm 1.1 13.7 ± 1.1 \pm 1.1 13.6 ± 0.5 \pm 0.5 1.6 ± 0.2 \pm 0.2 179.3 ± 4.0 \pm 4.0 19.9 ± 0.2 \pm 0.2 198.7 ± 5.0 \pm 5.0 6.7 ± 0.1 \pm 0.1 Direct Pred (32 layers) ✗ ✗ 100M 40.2 ± 1.3 \pm 1.3 13.6 ± 1.1 \pm 1.1 13.1 ± 0.4 \pm 0.4 1.6 ± 0.2 \pm 0.2 174.0 ± 18.0 \pm 18.0 19.1 ± 1.7 \pm 1.7 227.7 ± 34.5 \pm 34.5 6.5 ± 1.9 \pm 1.9 Looped TF ✓ ✗ 7M 68.4 ± 3.7 \pm 3.7 23.6 ± 1.9 \pm 1.9 50.0 ± 7.6 \pm 7.6 6.2 ± 3.2 \pm 3.2 136.0 ± 16.1 \pm 16.1 20.5 ± 1.5 \pm 1.5 157.3 ± 9.0 \pm 9.0 7.2 ± 0.7 \pm 0.7 HRM ✓ ✗ 27M 78.7 ± 2.9 \pm 2.9 26.7 ± 1.3 \pm 1.3 37.4 ± 0.3 \pm 0.3 4.7 ± 0.1 \pm 0.1 109.7 ± 1.5 \pm 1.5 21.8 ± 0.3 \pm 0.3 164.3 ± 21.6 \pm 21.6 8.9 ± 1.7 \pm 1.7 TRM ✓ ✗ 7M 66.8 ± 5.7 \pm 5.7 36.1 ± 22.5 \pm 22.5 17.5 ± 11.2 \pm 11.2 2.0 ± 1.3 \pm 1.3 109.3 ± 3.1 \pm 3.1 22.3 ± 0.6 \pm 0.6 170.7 ± 17.9 \pm 17.9 6.8 ± 0.3 \pm 0.3 AR ✗ ✓ 10.6M 96.3 ± 1.0 \pm 1.0 84.8 ± 0.8 \pm 0.8 90.0 ± 2.2 \pm 2.2 53.2 ± 0.8 \pm 0.8 19.0 ± 11.3 \pm 11.3 83.0 ± 0.7 \pm 0.7 61.3 ± 8.3 \pm 8.3 40.0 ± 0.3 \pm 0.3 MDLM ✗ ✓ 12.6M 96.1 ± 1.5 \pm 1.5 87.2 ± 0.6 \pm 0.6 74.3 ± 6.6 \pm 6.6 47.4 ± 2.2 \pm 2.2 2.7 ± 0.6 \pm 0.6 84.5 ± 4.0 \pm 4.0 12.0 ± 7.0 \pm 7.0 48.2 ± 1.4 \pm 1.4 GRAM (Ours) ✓ ✓ 10M 99.7 ± 0.3 \pm 0.3 90.3 ± 1.9 \pm 1.9 89.7 ± 2.7 \pm 2.7 57.5 ± 3.4 \pm 3.4 2.7 ± 2.1 \pm 2.1 85.8 ± 0.5 \pm 0.5 3.3 ± 1.5 \pm 1.5 51.3 ± 2.8 \pm 2.8

### 4.3 Exploring GRAM as an Unconditional Generator