Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning — part 5/7

Cleaned-text code points [58658, 74575). Bibliography omitted here; source.md includes it.
Tab. 9 gives the symbolic cost accounting behind these choices. The key distinction is between the length of the forward trajectory and the length of the backward graph. Let T T denote the number of outer-loop steps covered by a long trajectory. We write C ⁡ ( ⋅ ) C(\cdot) for the backward and parameter-update cost per optimizer interval and M ⁡ ( ⋅ ) M(\cdot) for the retained training memory per optimizer interval. The resulting accounting separates the effects of detached carry and truncation, which reduce backward and memory pressure, from SOT, which changes when the optimizer update happens. A more detailed derivation of this symbolic cost accounting is available in a separate blog post ( Huang, 2026 ) . For the remaining local terms, we use: Backward and parameter-update costs c ℓ c_{\ell} One local loss/head backward. c B c_{B} Segment parameter-backward. c B trunc c_{B}^{\mathrm{trunc}} Truncated segment parameter-backward. c J c_{J} Temporal state-backward through a segment. c θ c_{\theta} Extra shared-gradient accumulation. c u c_{u} Parameter update for one recurrent block. Retained memory terms a f a_{f} Activation memory for one segment. a ℓ a_{\ell} Activation memory for one loss/head branch. a f det a_{f}^{\mathrm{det}} Segment activation after detached carry. a f trunc a_{f}^{\mathrm{trunc}} Segment activation under truncation. P P Parameter-side memory for one recurrent block. These terms come from the local loop backward equations. For one segment transition 𝐳 s + 1 = f θ  ( 𝐳 s , 𝐱 ) \mathbf{z}_{s+1}=f_{\theta}(\mathbf{z}_{s};\mathbf{x}) , define J s \displaystyle J_{s} = ∂ 𝐳 s + 1 ∂ 𝐳 s , \displaystyle=\frac{\partial\mathbf{z}_{s+1}}{\partial\mathbf{z}_{s}}, B s \displaystyle B_{s} = ∂ 𝐳 s + 1 ∂ θ , \displaystyle=\frac{\partial\mathbf{z}_{s+1}}{\partial\theta}, (12) 𝐳 ¯ s \displaystyle\bar{\mathbf{z}}_{s} = J s ⊤  𝐳 ¯ s + 1 , \displaystyle=J_{s}^{\top}\bar{\mathbf{z}}_{s+1}, ∇ θ ( s )  ℒ \displaystyle\nabla_{\theta}^{(s)}\mathcal{L} = B s ⊤  𝐳 ¯ s + 1 , \displaystyle=B_{s}^{\top}\bar{\mathbf{z}}_{s+1}, where the second line uses column adjoints and 𝐳 ¯ s + 1 = d  ℒ / d  𝐳 s + 1 \bar{\mathbf{z}}_{s+1}=d\mathcal{L}/d\mathbf{z}_{s+1} is the incoming adjoint. Thus c J c_{J} denotes the temporal state-backward vector–Jacobian product through J s J_{s} , while c B c_{B} denotes the segment parameter-backward vector–Jacobian product through B s B_{s} . For shared recurrent weights, the step-local parameter contributions ∇ θ ( s )  ℒ \nabla_{\theta}^{(s)}\mathcal{L} are accumulated into a shared gradient buffer, which gives the c θ c_{\theta} term. These equations are a notation device for cost accounting; standard autograd need not materialize J s J_{s} or B s B_{s} explicitly. With this notation, full-gradient training through a weight-tied trajectory has C full  ( T ) \displaystyle C_{\mathrm{full}}(T) = c ℓ + T  c B + ( T − 1 )  c J + ( T − 1 )  c θ + c u , \displaystyle=c_{\ell}+Tc_{B}+(T-1)c_{J}+(T-1)c_{\theta}+c_{u}, (13) M full  ( T ) \displaystyle M_{\mathrm{full}}(T) = T  a f + a ℓ + P . \displaystyle=Ta_{f}+a_{\ell}+P. (14) Detached carry keeps the forward trajectory long but removes the temporal state-backward chain. For terminal-loss training with detached carry, only the final local transition contributes to the backward graph: C det  ( T ) \displaystyle C_{\mathrm{det}}(T) = c ℓ + c B + c u , \displaystyle=c_{\ell}+c_{B}+c_{u}, (15) M det  ( T ) \displaystyle M_{\mathrm{det}}(T) = a f det + a ℓ + P . \displaystyle=a_{f}^{\mathrm{det}}+a_{\ell}+P. (16) SOT then changes the optimizer interval itself from a full T T -step trajectory to one outer-loop step: C SOT \displaystyle C_{\mathrm{SOT}} = c ℓ + c B + c u , \displaystyle=c_{\ell}+c_{B}+c_{u}, M SOT \displaystyle M_{\mathrm{SOT}} = a f det + a ℓ + P , \displaystyle=a_{f}^{\mathrm{det}}+a_{\ell}+P, (17) C SOT + trunc \displaystyle C_{\mathrm{SOT+trunc}} = c ℓ + c B trunc + c u , \displaystyle=c_{\ell}+c_{B}^{\mathrm{trunc}}+c_{u}, M SOT + trunc \displaystyle M_{\mathrm{SOT+trunc}} = a f trunc + a ℓ + P . \displaystyle=a_{f}^{\mathrm{trunc}}+a_{\ell}+P. (18)

Notes. The local symbols follow the surrounding text and the derivation in Huang (2026) . Only the final local transition contributes to the backward graph in the terminal-loss detached-carry row. The trajectory-supervision row uses the offline end-of-trajectory loss accumulation contract. Depth-independent constants outside the recurrent chain are omitted; a fixed 16  L 16L -step training horizon is obtained by multiplying by the number of intervals needed to cover 16  L 16L outer-loop steps.

### A.3 Additional Experiments on Randomized State Initialization

This section provides supporting ablations for randomized state initialization ( RI ) in Sec. 5.1 . The main text uses simple zero-mean Gaussian initial states to improve coverage over attractor basins under depth–breadth scaling. Here we ask two narrower questions: whether replacing this simple stochastic prior with an input-conditioned learnable initializer is beneficial, and how sensitive the method is to the Gaussian noise scale. The results support the main-text design choice: simple randomized initialization is effective, while the tested learnable initializer and scale tuning do not change the central conclusion.

#### Learnable initial state.

A body of work on diffusion models suggests that the choice of initialization can significantly affect generation quality, and that learning a better initialization than standard Gaussian noise can be beneficial. For example, Zhou et al. (2025) proposes golden noise , where an auxiliary network learns to transform Gaussian noise into a prompt-conditioned initialization that improves alignment. Related approaches include directly optimizing the initial noise at inference time ( Guo et al., 2024 ) , as well as reducing the initialization gap in video diffusion models through structured initialization schemes ( Wu et al., 2024 ) . These results motivate a natural question in our setting: whether learning the initial latent state 𝐳 0 \mathbf{z}_{0} can similarly improve iterative reasoning models.

Instead of sampling 𝐳 0 ∼ 𝒩 ⁡ ( 0 , σ 0  I ) \mathbf{z}_{0}\sim\mathcal{N}(0,\sigma_{0}I) , we consider a simple learnable initialization scheme. Specifically, we introduce an input-conditioned 2-layer MLP g ϕ g_{\phi} that predicts the initial state 𝐳 0 = g ϕ  ( 𝐱 ) \mathbf{z}_{0}=g_{\phi}(\mathbf{x}) , and train ϕ \phi jointly with the rest of the model parameters under the same training objective. This can be viewed as learning a conditional prior over the initial latent state.

#### Empirical observation.

In our experiments, learning the initialization does not improve the training or held-out evaluation accuracy curves under the protocol used here. Tab. 10 reports the held-out exact accuracy of the TRM baseline, TRM with randomized state initialization (TRM + RI), and TRM with a learnable initializer 𝐳 0 = g ϕ  ( 𝐱 ) \mathbf{z}_{0}=g_{\phi}(\mathbf{x}) . The learnable initializer reaches 83.99 % 83.99\% exact accuracy at 50k training steps, compared with 86.03 % 86.03\% for TRM + RI and 84.06 % 84.06\% for the TRM baseline. Thus, the tested input-conditioned initializer does not improve over simple randomized initialization, and it also does not provide a reliable gain over the baseline TRM.

Across the full set of evaluated checkpoints up to 50k training steps, the learnable initializer never exceeds TRM + RI on held-out exact accuracy, and its best checkpoint remains slightly below the 50k result of the TRM baseline.

#### Scope and limitations.

Since we do not conduct further analysis or ablations on alternative initialization parameterizations, architecture designs, objectives, or regularization strategies, we do not make stronger claims about the general effectiveness of learnable initializations in iterative reasoning models. A more systematic study of initialization priors in latent state space is left for future work.

#### Randomness scale.

In the main text, we instantiate RI with zero-mean Gaussian initial states. This section examines how the scale of this initialization randomness affects TRM performance.

TRM maintains two latent spaces, a high-level latent 𝐳 H \mathbf{z}_{H} and a low-level latent 𝐳 L \mathbf{z}_{L} . We therefore vary the noise scale for each latent separately by sampling 𝐳 H ∼ 𝒩 ⁡ ( 0 , σ H  I ) \mathbf{z}_{H}\sim\mathcal{N}(0,\sigma_{H}I) and 𝐳 L ∼ 𝒩 ⁡ ( 0 , σ L  I ) \mathbf{z}_{L}\sim\mathcal{N}(0,\sigma_{L}I) . When a noise scale is not explicitly specified, we use the default σ = 1 \sigma=1 . Setting σ = 0 \sigma=0 corresponds to a deterministic (fixed) initialization.

Overall, randomized state initialization can improve performance over the deterministic fixed-initialization run in this sweep, and the choice of noise scale matters. For example, randomizing only 𝐳 H \mathbf{z}_{H} (setting σ H = 1 , σ L = 0 \sigma_{H}=1,\sigma_{L}=0 ) improves exact accuracy from 84.06 % 84.06\% to 86.29 % 86.29\% , and randomizing only 𝐳 L \mathbf{z}_{L} (setting σ H = 0 , σ L = 1 \sigma_{H}=0,\sigma_{L}=1 ) improves it to 86.25 % 86.25\% . For the joint sweep, the evaluated settings with σ H = 1 \sigma_{H}=1 achieve exact accuracies 86.03 % 86.03\% , 86.83 % 86.83\% , 87.30 % 87.30\% , and 86.85 % 86.85\% for σ L ∈ { 1 , 4 , 8 , 16 } \sigma_{L}\in\{1,4,8,16\} , respectively. Fixing σ L = 1 \sigma_{L}=1 and increasing σ H \sigma_{H} gives 86.03 % 86.03\% , 86.38 % 86.38\% , 86.08 % 86.08\% , and 86.29 % 86.29\% for σ H ∈ { 1 , 4 , 8 , 16 } \sigma_{H}\in\{1,4,8,16\} . Among the tested settings, the best performance is achieved by combining moderate noise on 𝐳 H \mathbf{z}_{H} with a larger noise scale on 𝐳 L \mathbf{z}_{L} , peaking at σ H = 1 , σ L = 8 \sigma_{H}=1,\sigma_{L}=8 with exact accuracy 87.30 % 87.30\% . These observations are consistent with the view that initialization randomness is most helpful when it places the iterative dynamics within basins that lead to correct solutions, whereas overly large perturbations may not further improve (and can slightly reduce) accuracy.

Due to limited computational resources, we report results from a single run for each configuration and do not include variance estimates across multiple random seeds. A more systematic study of variability across runs and a denser sweep of noise scales are left for future work.

### A.4 Additional Experiments on Path Stochasticity

As shown in Eq. 2 , we introduce path stochasticity by injecting step-wise noise into the iterative update, 𝐳 k + 1 = 𝐳 k + ( 1 − λ )  r θ  ( 𝐳 k , 𝐱 ) + β  ε k \mathbf{z}_{k+1}=\mathbf{z}_{k}+(1-\lambda)\,r_{\theta}(\mathbf{z}_{k};\mathbf{x})+\beta\,\varepsilon_{k} , where ε k ∼ 𝒩 ⁡ ( 0 , I ) \varepsilon_{k}\sim\mathcal{N}(0,I) by default.

Tab. 12 reports a brief ablation over the fixed Gaussian noise scale β \beta , along with a learned-noise variant. Moderate Gaussian noise yields comparable performance to the default setting, whereas overly large noise can slightly reduce evaluation accuracy. Although the learned-noise variant performs best in this small ablation, Tab. 13 shows that it does not consistently improve when the number of stochastic samples or restarts S S is increased. We therefore use fixed Gaussian noise by default and keep learned noise as an ablation rather than adding extra noise parameters to the main method.

### A.5 Generalization Beyond the Main Setting

We further test whether the proposed training-and-scaling recipe is tied to the main Sudoku-Extreme and Maze-Unique settings. Tab. 14 reports two complementary checks: performance on Mini-ARC ( Kim et al., 2022 ) , and transfer from the Sudoku MLP-token-mixer backbone to a self-attention Transformer backbone.

On Mini-ARC, EqR reaches 55.28 % 55.28\% exact accuracy, improving over both HRM ( 44.85 % 44.85\% ) and TRM ( 48.35 % 48.35\% ). On Sudoku-Extreme, the same pattern appears across two token mixers. For the MLP-token-mixer backbone used in the main Sudoku setting, the training interventions improve accuracy from 84.1 % 84.1\% to 86.4 % 86.4\% , and inference scaling raises it further to 99.8 % 99.8\% . For the Transformer backbone, the corresponding numbers are 72.0 % 72.0\% , 74.7 % 74.7\% , and 95.9 % 95.9\% . These results suggest that the gains are not limited to the main Sudoku-Extreme and Maze-Unique experiments, nor to the specific MLP-token-mixer backbone used in the standard Sudoku-Extreme setup.

### A.6 Seed Stability Diagnostics

We check whether the same-budget gain is stable across independent random seeds for EqR. Across five seeds at 50k training steps, the baseline reaches 84.33 ± 0.59 % 84.33\pm 0.59\% exact accuracy (95% CI: [83.59, 85.07]), whereas EqR reaches 86.18 ± 0.44 % 86.18\pm 0.44\% (95% CI: [85.63, 86.72]). Thus, EqR remains higher than the baseline at the same budget and shows slightly lower seed-to-seed variation.

## Appendix B Qualitative Study

Fig. 13 visualizes one TRM reasoning trajectory on a Sudoku-Extreme puzzle. Each panel decodes the model’s current latent state into a full Sudoku grid after one iteration step. Early iterations already propose many correct entries, but later iterations continue to revise both correct and incorrect cells. The circled cell illustrates this non-monotonic behavior: the decoded value alternates across several candidates before the trajectory eventually settles into a consistent solution.

## Appendix C Dataset Details and Task Definitions

This appendix defines the dataset variants and task conventions used throughout our experiments, with particular attention to the distinction between the original Maze-1k benchmark and our uniquely solvable Maze-Unique setting.

### C.1 Dataset and Benchmark Specifications

#### Sudoku-Extreme

Sudoku-Extreme ( Wang et al., 2025 ) is a benchmark of exceptionally challenging 9 × 9 9\times 9 Sudoku instances designed to stress long-horizon constraint satisfaction. According to the HRM paper, it remains difficult even for strong modern reasoning models such as DeepSeek-R1 and Claude 3.7 8k. We reuse the code and data released with the Sudoku-Extreme dataset from HRM ( Wang et al., 2025 ) .

Sudoku-Lite. As shown in Tab. 16(a) , the validation set of Sudoku-Extreme is very large. To improve evaluation efficiency, we introduce a subset of 2048 cases sampled uniformly at random from Sudoku-Extreme, which we term Sudoku-Lite.

Tab. 15 compares exact accuracy on Sudoku-Extreme and Sudoku-Lite. Across multiple model variants, performance on Sudoku-Lite is slightly worse, suggesting that this smaller evaluation subset is not an easier benchmark for current models.

Dataset TRM HRM EqR Sudoku-Extreme 84.1 61.0 86.4 Sudoku-Lite 82.0 57.8 84.3

Maze-1k. Maze-1k evaluates shortest-path prediction on a 30 × 30 30\times 30 grid with obstacle cells. The benchmark follows the instance-generation procedure of Lehnert et al. (2024) , as used by HRM. Many instances admit multiple shortest paths, while the released dataset provides one labeled path per input; Fig. 14(a) shows a representative example.

Maze-Unique. We construct Maze-Unique as a controlled 30 × 30 30\times 30 variant in which every retained instance has a unique shortest path. Each maze is generated as a perfect maze, i.e., a tree-structured grid with a single simple path between any pair of cells. For each maze, we repeatedly sample start-goal pairs, compute the shortest-path length, retain pairs within the target length range, and de-duplicate mazes across the training and test splits. The final dataset contains 1,000 training instances and 1,000 test instances. As shown in Tab. 16(b) , this filtering does not make the task shorter by path length: Maze-Unique has a slightly larger average shortest-path length than Maze-1k. A concrete example is shown in Fig. 14(b) .

#### Naming convention.