Generative Recursive Reasoning — part 3/5

Cleaned-text code points [31046, 46990). Bibliography omitted here; source.md includes it.

Setup. To investigate GRAM’s unconditional generative capability beyond conditional reasoning, we evaluate generation in two domains: structured constraint generation on Sudoku (from empty boards, evaluated by the fraction of generated boards satisfying Sudoku constraints) and image generation on binarized MNIST [ 15 ] , where pixel values are thresholded to 0 0 or 1 1 (evaluated by Inception Score (IS) [ 42 ] and FID [ 43 ] ). In both cases, the input is replaced by an empty conditioning signal and the model samples an output from its learned prior. Baselines include D3PM [ 44 ] , a discrete diffusion model, on both tasks, and additionally a VAE [ 45 ] trained with binary reconstruction loss on MNIST. To ensure a fair comparison with existing literature, FID is calculated using real samples from the original standard MNIST.

Generative Behavior Beyond Reasoning. GRAM extends from conditional reasoning to unconditional generation in two different domains. On Sudoku generation ( Figure 5 ), GRAM produces valid boards with 99.05% validity using 10.9M parameters and 16 supervision steps, surpassing D3PM baselines that use up to 55.1M parameters and 1000 denoising steps. Figure 7 shows qualitative examples, illustrating that the model produces diverse, fully valid boards from empty inputs without any explicit constraint checker. On MNIST ( Table 2 ), the deterministic baseline TRM exhibits mode collapse (FID 303.29), whereas GRAM produces recognizable digits with IS and FID comparable to D3PM. Together, these results indicate that GRAM’s stochastic latent transitions support generative modeling beyond symbolic reasoning, with constraint satisfaction emerging as a natural byproduct of the recursive generative process.

Inference-Time Scaling Transfers to Generation. Table 2 further shows that increasing recursion at inference improves generation quality monotonically (IS → 2.04 1.85\!\to\!2.04 , FID → 73.34 84.08\!\to\!73.34 from 8 to 256 steps), even though training uses only 16 steps. This indicates that the iterative-refinement advantage of recursive models carries over into the generative regime. Figure 6 visualizes this process; additional samples are in Section D.4 .

### 4.4 Ablation Study

We ablate key design choices of GRAM on Sudoku-Extreme and N-Queens ( 8 × 8 8\times 8 ) using 5 samples. Table 3 summarizes the results.

Stochastic Guidance Provides Consistent Gains Across Architectures. Table 3(a) shows that stochastic guidance (SG) improves performance regardless of the underlying architecture: SG alone lifts the flat Looped TF baseline, and combining SG with deep supervision already reaches 100% on N-Queens. The full GRAM (with hierarchical recursion on top) achieves the best results overall (93.96% / 99.69%). While the effect of hierarchical recursion is task-dependent, SG yields consistent gains in every configuration, supporting our design of stochastic guidance as the core extension introduced by GRAM.

Both Stochasticity and Guidance Are Essential. We ablate each component by modifying the learned distribution ϵ t ∼ 𝒩 ⁡ ( μ θ , σ θ 2 ​ I ) \epsilon_{t}\sim\mathcal{N}(\mu_{\theta},\sigma^{2}_{\theta}I) in Equation 4 . Removing guidance ( 𝒩 ⁡ ( 0 , σ θ 2 ​ I ) \mathcal{N}(0,\sigma^{2}_{\theta}I) ) maintains Sudoku performance (94.88%), indicating that stochasticity alone can enable diverse reasoning paths. However, this variant collapses on N-Queens (50.27%), where structured guidance is necessary to navigate multi-solution spaces. Removing stochasticity ( 𝒩 ⁡ ( μ θ , 0 ) \mathcal{N}(\mu_{\theta},0) ) fails completely (0.0% on both tasks), as deterministic guidance conditioned on the target leads to severe overfitting.

Naive Stochasticity Does Not Help TRM. We test two simple approaches to add stochasticity to TRM: (1) stochastic decoding , which samples from the output distribution instead of argmax, and (2) random initialization , which samples z 0 z_{0} from a Gaussian 𝒩 ⁡ ( 0 , I ) \mathcal{N}(0,I) at each inference. Neither improves performance, demonstrating that GRAM’s gains stem from the variational framework rather than mere randomness.

## 5 Conclusions and Limitations

We introduced GRAM, a generative framework that transforms deterministic recursive architectures into probabilistic generative models capable of modeling both p ⁡ ( y ∣ x ) p(y\mid x) and p ⁡ ( x ) p(x) via recursive amortized variational inference. For reasoning problems, introducing stochasticity into latent transitions enables diverse solution discovery and improved exploration compared to deterministic counterparts. Notably, we demonstrate GRAM can leverage width-based inference-time scaling as a complement to depth: by sampling multiple latent trajectories in parallel, bypassing the latency bottleneck of depth-only scaling. Our ablations further reveal that stochastic guidance is a general-purpose extension that consistently improves any recursive architecture, and that the gains stem specifically from the variational framework — not from mere randomness, as naive stochastic alternatives applied to existing models yield no improvement.

Beyond solution-seeking, GRAM also demonstrates potential as an unconditional generative model through recursion-based generation over inputs, with generation quality improving monotonically with recursive depth even beyond training-time steps. This suggests new directions for generative modeling via hierarchical recursion. Despite these strengths, the sequential nature of deep supervision limits training efficiency compared to Transformers, posing a significant barrier to scaling GRAM toward larger foundation models.

## Acknowledgment

This research was supported by the Brain Pool Plus Program (No. 2021H1D3A2A03103645) and the GRDC (Global Research Development Center) Cooperative Hub Program (RS-2024-00436165) through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT (MSIT). This work was also supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2024-00509279, Global AI Frontier Lab) and by the NYU-KAIST Global Innovation and Research Institute. Minsu Kim acknowledges funding from the KAIST Jang Young Sil Fellow Program. We are especially grateful to Gyubin and Seungju for their non-trivial contributions, and we thank the members of the MLML for valuable discussions and feedback throughout this project.

## Broader Impacts

GRAM studies probabilistic recursive reasoning for structured reasoning and generation. By maintaining multiple latent trajectories, it may benefit tasks such as constraint satisfaction, and scientific problem solving, where uncertainty and multiple valid solutions are common. It also suggests a way to improve reasoning through inference-time computation rather than parameter scaling alone. Its generality also entails risks: plausible but invalid generations may be mistaken for verified solutions in downstream decision-making pipelines, and multi-sample inference may increase computational and energy costs at scale. Since our experiments focus on controlled benchmarks, deployment in real-world or high-stakes settings would require rigorous validation, uncertainty calibration, and domain-specific safeguards.

## Appendix A Additional Method Details

### A.1 Adaptive Computation Time

GRAM optionally adopts adaptive computation time (ACT) [ 8 , 9 , 10 ] at inference, allowing each trajectory to terminate at a learned halting depth rather than running for a fixed number of supervision steps. We follow the Q-learning formulation introduced by HRM [ 8 ] and adopted in TRM [ 9 ] .

Halt head. The decoder includes an auxiliary head q ψ : ℝ D → ℝ 2 q_{\psi}:\mathbb{R}^{D}\to\mathbb{R}^{2} that maps the high-level state h h to two scalar values, q ψ ​ ( h ) = ( q halt , q continue ) q_{\psi}(h)=(q^{\mathrm{halt}},q^{\mathrm{continue}}) . These are interpreted as estimated Q-values for the binary action of halting or continuing computation at the current supervision step.

Training. The halt head is trained jointly with the main objective via a temporal-difference loss. After computing the latent state z T ( n ) z_{T}^{(n)} at the end of supervision step n n , we form Q-learning targets: • q ^ n halt = 𝟏 [ y ^ ( n ) = y ] \hat{q}^{\mathrm{halt}}_{n}=\mathbf{1}[\hat{y}^{(n)}=y] , indicating whether decoding the current state would yield a correct prediction.

• q ^ n continue = max ⁡ ( q n + 1 halt , q n + 1 continue ) \smash{\hat{q}^{\mathrm{continue}}_{n}=\max\!\left(q^{\mathrm{halt}}_{n+1},\,q^{\mathrm{continue}}_{n+1}\right)} , the bootstrapped value of running one more supervision step.

The halt head is trained by regression to these targets:

This auxiliary loss is added to the main training objective and contributes only through the halt head; it does not propagate gradients into the recursive core.

Inference. At inference, computation proceeds one supervision step at a time. After each step n n , we evaluate q ψ ​ ( h ( n ) ) q_{\psi}(h^{(n)}) and halt if q n halt > q n continue q^{\mathrm{halt}}_{n}>q^{\mathrm{continue}}_{n} , returning y ^ ( n ) \hat{y}^{(n)} as the prediction. Otherwise, computation continues to the next supervision step, up to a maximum budget of N sup max N_{\mathrm{sup}}^{\max} steps. Different trajectories sampled in parallel may therefore terminate at different depths, complementing the parallel-sampling scheme described in Section 2.3 . In practice, we found that using only q halt q^{\mathrm{halt}} (halting when σ ⁡ ( q halt ) > 0.5 \sigma(q^{\mathrm{halt}})>0.5 , without the continue branch) performs comparably while simplifying implementation; our released code uses this variant.

### A.2 Latent Process Reward Model (LPRM).

To rank or select among sampled candidates, we train a value head v ψ ​ ( z t ) v_{\psi}(z_{t}) to predict the expected accuracy of the final output, conditioned on the current latent state z t z_{t} . The LPRM is trained jointly with the main objective via a regression loss: ℒ LPRM = ∑ t = 1 T ( v ψ ​ ( z t ) − r ) 2 , \displaystyle\mathcal{L}_{\text{LPRM}}=\sum_{t=1}^{T}(v_{\psi}(z_{t})-r)^{2}, (16) where r ∈ [ 0 , 1 ] r\in[0,1] denotes the accuracy of the final prediction for a given trajectory.

### A.3 Empirical Validation of the Surrogate Objective

We further analyze the approximation introduced by the surrogate training objective ℒ GRAM \mathcal{L}_{\mathrm{GRAM}} used in Section 2.2 , both qualitatively and empirically.

Truncation as a gradient approximation. We frame ℒ GRAM \mathcal{L}_{\mathrm{GRAM}} as a gradient approximation rather than a separate variational objective. The full trajectory-level ELBO ( Equation 13 ) involves a sum of KL terms across all T Total T_{\mathrm{Total}} transitions, and computing its exact gradient requires backpropagation through the entire trajectory. To enable training with constant memory, we propagate gradients only through the final transition of each supervision step. This is a standard practice in recurrent latent variable models with long computation chains: ELBOs over truncated sequences are used, for example, in VRNN [ 33 ] and SRNN [ 34 ] , while truncated latent imagination is used in Dreamer-family world models [ 37 , 38 ] . Trading a small gradient bias for training stability via local truncation is therefore well-precedented; what is specific to GRAM is applying this approximation at the level of recursive reasoning trajectories rather than temporal sequences.

Empirical validation. To verify that optimizing ℒ GRAM \mathcal{L}_{\mathrm{GRAM}} effectively drives improvement in the full variational bound, we compute both quantities on the validation set throughout training. The full ELBO ℒ ELBO \mathcal{L}_{\mathrm{ELBO}} is evaluated as in Equation 13 , summing the reconstruction term and KL contributions across all T Total T_{\mathrm{Total}} transitions; the surrogate objective is evaluated as the average of ℒ GRAM ( n ) \mathcal{L}_{\mathrm{GRAM}}^{(n)} over the N sup N_{\mathrm{sup}} supervision steps. Figure 8 reports the results on Sudoku-Extreme and N-Queens 8 × 8 8\times 8 .

Both ℒ ELBO \mathcal{L}_{\mathrm{ELBO}} and ℒ GRAM \mathcal{L}_{\mathrm{GRAM}} improve monotonically throughout training on both tasks. This indicates that, despite the truncation, gradient updates of ℒ GRAM \mathcal{L}_{\mathrm{GRAM}} effectively drive improvement in the full variational bound. Since ℒ ELBO \mathcal{L}_{\mathrm{ELBO}} also serves as an indirect estimate of the negative log-likelihood, its consistent improvement provides evidence that GRAM optimizes a well-defined data likelihood, even though training relies on the surrogate.

The gap between the two curves in Figure 8 reflects the structural difference between the two quantities — ℒ ELBO \mathcal{L}_{\mathrm{ELBO}} accumulates KL terms across all transitions while ℒ GRAM \mathcal{L}_{\mathrm{GRAM}} evaluates only the final-step KL of each supervision step — rather than an optimization failure. This gap is consistent with ℒ GRAM \mathcal{L}_{\mathrm{GRAM}} being a biased but useful surrogate for ℒ ELBO \mathcal{L}_{\mathrm{ELBO}} .

## Appendix B Training and Architecture Details

### B.1 Architecture Details

GRAM consists of three components: Encoder, Recursive Core, and Decoder.

Encoder. Input tokens are mapped to embeddings via a token embedding layer, optionally concatenated with puzzle embeddings (for ARC [ 13 , 14 ] ), and combined with positional encodings (RoPE) [ 46 ] . The embeddings are scaled by D \sqrt{D} and prepended with 16 puzzle embedding tokens [ 8 ] .

Recursive Core. The core maintains two latent states: h h (high-level) and l l (low-level). For each outer step, the low-level state is refined K K times via l ← f L ​ ( l , h + e x ) l\leftarrow f_{L}(l,h+e_{x}) , injecting the input embedding at each iteration. The high-level state is then updated via h ← f H ​ ( h , l ) h\leftarrow f_{H}(h,l) . Both f L f_{L} and f H f_{H} share the same architecture: a stack of attention and SwiGLU [ 47 ] MLP layers. In addition, as an exception, we use [SwiGLU + SwiGLU] network for the Recursive Core module instead of [Attention + SwiGLU] for Sudoku tasks, following [ 9 ] . For initialization of z 0 = ( h 0 , l 0 ) z_{0}=(h_{0},l_{0}) , we sample once from the standard Gaussian distribution 𝒩 ⁡ ( 0 , I ) \mathcal{N}(0,I) , then save the value within the network checkpoint and load it again, meaning the initialized z 0 z_{0} has a fixed value.

Decoder. The decoder extracts content tokens from h h (excluding puzzle embedding positions) and maps them to logits via a SwiGLU MLP head. An auxiliary head predicts halt decisions and correctness values from the first token of h h .

##### Encoder and Decoder for Image Patches.

In the MNIST [ 15 ] image generation task, we first construct a binarized dataset by normalizing the original discrete pixel values ( 0 ∼ 255 0\sim 255 ) to the continuous range [ 0 , 1 ] [0,1] and applying a threshold at 0.5 0.5 . For the network architecture, we employ a convolutional patch encoder, following [ 48 , 49 ] .

The encoding process proceeds in three stages. First, the discrete input tokens x ∈ { 0 , 1 } x\in\{0,1\} are normalized to the range [ − 1 , 1 ] [-1,1] . Second, to capture local spatial dependencies before patchification, the normalized image passes through a shallow convolutional encoder. This encoder consists of two stacked blocks, where each block comprises a 2D convolution [ 50 , 51 ] with a 5 × 5 5\times 5 kernel and padding 2, a SiLU non-linearity [ 52 ] , and Group Normalization (GN) [ 53 ] . Finally, the resulting feature map is divided into non-overlapping patches of size P × P P\times P and linearly projected to match the model’s hidden dimension D D . The detailed architectural specifications and dimension transitions are summarized in Table 5 .