Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning — part 4/7

Cleaned-text code points [46646, 58658). Bibliography omitted here; source.md includes it.

Tab. 8 ( 8(a) ) shows that introducing a weight-tied iterative model improves substantially over the vanilla feedforward baseline on Sudoku (2.6% → \rightarrow 32.6%); Fig. 9 further shows the corresponding training/evaluation error curves on Sudoku, under matched layer-evaluation budgets. The curves show a more nuanced trade-off than a simple train–evaluation gap: at its best-evaluation checkpoint, the 2-iteration weight-tied model reduces both training and evaluation error relative to the feedforward depth baseline; among iterative models, increasing the iteration steps then raises the training error while reducing the evaluation error. Tab. 6 records this checkpoint-level comparison for the four plotted runs.

Thus, a weight-tied parameterization creates useful iterative capacity: under the matched layer-evaluation budget, the feedforward model underperforms in both training and evaluation, whereas replacing independent layers with repeated applications of shared parameters yields better generalization and a smaller train–evaluation mismatch. However, this capacity remains insufficient under the current budget, motivating the depth-scaling and additional training-dynamics ablations below.

Iteration steps induce a trade-off between alignment and feasibility.

Larger iterations (depth) are desirable because they can move states closer to stable, solution-aligned attractors; however, they also make training less feasible. As shown in Tab. 8 ( 8(b) ), doubling the iteration depth is already helpful, improving Sudoku from 32.6% to 51.3%. The natural next step is to scale this trajectory to a much larger depth multiplier (e.g., 16 × 16\times ), but the full-gradient backpropagation through the long trajectory is memory-prohibitive and recurrent-gradient products can explode or vanish. Detaching the carried state before the terminal loss makes the 16 × 16\times run feasible: the backward graph covers only the final local transition, and with a stronger learning rate and lower weight decay this terminal-loss recipe with detached carry reaches 51.8%. However, the gain over the 2 × 2\times setting is marginal relative to the additional computation, indicating that simply extending the terminal-loss trajectory with detached carry is not an efficient way to leverage the potential of long iterations. This motivates a more careful choice of supervision placement and optimization schedule.

The local residual and implicit gradient conditions behind this interpretation are separated in Appendix A.1 ; here we focus on the training-dynamics evidence.

Segmented online training alternates latent-state and parameter updates.

The weak gain from terminal-loss training with detached carry suggests that feasibility is not the only issue: once gradients are detached along most of the trajectory, the intermediate carried states receive little direct supervision. A natural compensation is therefore to add loss anchors along the long trajectory, so that more carried states are trained while each backward graph remains local. As introduced in Sec. 3 , we call this trajectory supervision (an offline deep-supervision schedule): the model evaluates losses at multiple carried states, ℒ off ​ ( θ ) = 1 M ​ ∑ m = 1 M ℓ θ ​ ( 𝐳 t m , 𝐱 , 𝐲 ) , 𝐳 k + 1 = f θ ​ ( 𝐳 k , 𝐱 ) , \mathcal{L}_{\mathrm{off}}(\theta)=\frac{1}{M}\sum_{m=1}^{M}\ell_{\theta}(\mathbf{z}_{t_{m}};\mathbf{x},\mathbf{y}),\qquad\mathbf{z}_{k+1}=f_{\theta}(\mathbf{z}_{k};\mathbf{x}), (9) and updates the parameters only after the full trajectory has been processed. This keeps training cost controlled and helps restore supervision along the trajectory.

However, offline trajectory supervision creates a stale-trajectory mismatch. With parameters held fixed while anchors are collected, several transient lower-level iterates are all asked to match the same upper-level target before the operator has been updated. The resulting objective can behave like average-state matching: it rewards moving an aggregate of those states toward the target, even when that aggregate lies off the trajectory reachable by the updated operator. Thus, extra anchors provide more local supervision, but they can also pull the learned dynamics toward a target that is not reachable under the updated operator.

SOT instead alternates the two levels. For a segment of h h function evaluations, 𝐳 s \mathbf{z}_{s} denotes the detached carried state from the previous segment, while the current segment remains gradient-tracked: 𝐳 ~ s + 1 ​ ( θ ) \displaystyle\tilde{\mathbf{z}}_{s+1}(\theta) = f θ ( h ) ​ ( 𝐳 s , 𝐱 ) , \displaystyle=f_{\theta}^{(h)}(\mathbf{z}_{s};\mathbf{x}), (10) g s \displaystyle g_{s} = ∇ θ ℓ θ ​ ( 𝐳 ~ s + 1 ​ ( θ ) , 𝐱 , 𝐲 ) | θ = θ s , \displaystyle=\nabla_{\theta}\ell_{\theta}(\tilde{\mathbf{z}}_{s+1}(\theta);\mathbf{x},\mathbf{y})\big|_{\theta=\theta_{s}}, θ s + 1 \displaystyle\theta_{s+1} = θ s − η g s , 𝐳 s + 1 = stopgrad ( 𝐳 ~ s + 1 ( θ s ) ) . \displaystyle=\theta_{s}-\eta g_{s},\qquad\mathbf{z}_{s+1}=\operatorname{stopgrad}(\tilde{\mathbf{z}}_{s+1}(\theta_{s})). The next latent segment is then generated under the updated parameters θ s + 1 \theta_{s+1} . Each segment first takes a lower-level corrector step in latent space, then takes an upper-level parameter step that reshapes the operator. This alternating view is natural because changing θ \theta changes the attractor landscape, while advancing 𝐳 \mathbf{z} reveals which basin the current operator can actually reach. If the h h -step local solver contracts errors by a factor ρ < 1 \rho<1 near the current attractor and the local attractor map has sensitivity κ θ ≈ ∥ ( I − J 𝐳 , s ) − 1 ​ J θ , s ∥ \kappa_{\theta}\approx\lVert(I-J_{\mathbf{z},s})^{-1}J_{\theta,s}\rVert , where J 𝐳 , s J_{\mathbf{z},s} and J θ , s J_{\theta,s} are the corresponding local Jacobians at segment s s , then the carried-state tracking error e s = ∥ 𝐳 s − 𝐳 θ s ⋆ ∥ e_{s}=\lVert\mathbf{z}_{s}-\mathbf{z}^{\star}_{\theta_{s}}\rVert obeys the bound e s + 1 \displaystyle e_{s+1} = ∥ f θ s ( h ) ​ ( 𝐳 s , 𝐱 ) − 𝐳 θ s + 1 ⋆ ∥ \displaystyle=\lVert f_{\theta_{s}}^{(h)}(\mathbf{z}_{s};\mathbf{x})-\mathbf{z}^{\star}_{\theta_{s+1}}\rVert (11) ≤ ∥ f θ s ( h ) ​ ( 𝐳 s , 𝐱 ) − 𝐳 θ s ⋆ ∥ + ∥ 𝐳 θ s ⋆ − 𝐳 θ s + 1 ⋆ ∥ \displaystyle\leq\lVert f_{\theta_{s}}^{(h)}(\mathbf{z}_{s};\mathbf{x})-\mathbf{z}^{\star}_{\theta_{s}}\rVert+\lVert\mathbf{z}^{\star}_{\theta_{s}}-\mathbf{z}^{\star}_{\theta_{s+1}}\rVert ≲ ρ ​ e s + κ θ ​ ∥ θ s + 1 − θ s ∥ . \displaystyle\lesssim\rho e_{s}+\kappa_{\theta}\lVert\theta_{s+1}-\theta_{s}\rVert.

This separates the two effects: latent correction reduces the first term, while the parameter update contributes the attractor shift in the second term. Compared with trajectory supervision under detached carry, SOT therefore keeps supervision closer to the currently reachable trajectory instead of optimizing many stale anchors before any parameter update occurs.

The diagnostics below test this interpretation. They first show that early anchors can conflict with the detached trajectory, and then show that SOT turns the same long-rollout supervision into a much more effective training procedure.

Together, Fig. 11 and Tab. 7 show that intermediate anchors are not uniformly harmful: anchors over the full trajectory include early transient states and reduce accuracy from 51.8% to 47.1% relative to terminal-loss supervision, whereas anchors restricted to the late trajectory can improve performance. We interpret the late-anchor gain through the attractor-alignment view: if solution alignment is mediated by attractor basins, the same target loss has a different meaning before and after the trajectory reaches the basin. The residual trace gives a concrete proxy for this split: after the sharp early drop, later states lie on a more stable part of the rollout, so supervising them creates less conflict with the model’s own dynamics.

Supervision Anchors Acc. terminal loss 16 16 51.80 full anchors 1 : 16 1{:}16 47.10 late anchors 8 : 16 8{:}16 51.36 late anchors 12 : 16 12{:}16 57.50

Before the trajectory enters a solution-aligned basin, supervision on intermediate states mainly acts as coarse guidance, and its local gradients can be unreliable because they are attached to transient states rather than to a stable solution. After the trajectory enters the basin, the same target supervision becomes more trustworthy because the local state is already near the solution attractor. Late anchors therefore concentrate gradient signal in the regime where the gradients are aligned with the desired attractor. Under this view, if several late anchors all lie in the attractor basin, accumulating or upweighting those late losses behaves like a larger effective step size on reliable gradients, instead of amplifying noisy early-trajectory gradients. For each anchor range in Tab. 7 , we report the best accuracy over learning rates { 10 − 3 , 5 × 10 − 4 , 10 − 4 , 5 × 10 − 5 } \{10^{-3},5{\times}10^{-4},10^{-4},5{\times}10^{-5}\} and weight decays { 0.1 , 0.5 , 1.0 } \{0.1,0.5,1.0\} .

The SOT rows give the direct evidence for the alternating formulation. As shown in Tab. 8 ( 8(c) ), switching from trajectory supervision at 16 × 16\times to SOT at 16 × 16\times improves Sudoku from 47.1% to 74.7% under the same nominal depth multiplier. The gain is therefore not explained by adding anchors alone: the key change is that parameter updates are interleaved with latent-state updates, so later trajectory segments are generated by the current operator rather than by stale parameters. SOT can further reduce memory and compute cost when combined with in-segment gradient truncation, which shortens the backward graph inside each segment. Tab. 8 ( 8(d) ) shows that this truncation interacts strongly with the latent structure. In the single-latent setting, in-segment truncation reduces Sudoku-Extreme accuracy from 74.7% to 67.2%. With hierarchical iterations, however, the same truncation improves accuracy from 69.8% to 75.4%. This reversal suggests that hierarchical iterations shape the training dynamics differently from the single-latent update, an interaction we discuss in the next finding.

Hierarchical iterations change performance, but their effect is difficult to decouple from the training recipe.

Hierarchical iterations interact with other training strategies. As shown in Tab. 8 ( 8(d) ), without in-segment truncation, the single-latent model is stronger on Sudoku, whereas with truncation the hierarchical variant becomes stronger. The Adaptive Computation Time (ACT) ablation rows in Tab. 8 ( 8(e) ) show a second interaction with the halting mechanism. With learned ACT, the hierarchical latent model reaches 84.8% on Sudoku, while the corresponding single-latent 𝐳 \mathbf{z} variant reaches 73.9%. Thus, hierarchy is not a standalone switch; its effect depends on the surrounding training recipe. We also observed that the relative performance of hierarchical iterations compared with the single-latent model depends on the task.

Adaptive Computation Time (ACT) changes training dynamics through learned halting.

The halting mechanism is not only for efficiency: when applied to the training of iterative models, it also shapes the training dynamics, and different halting signals lead to different outcomes. As shown in Tab. 8 ( 8(e) ), oracle (ground-truth-based) halting collapses the hierarchical model from 75.4% to 13.6% on Sudoku, whereas a learned ACT head improves the result from 75.4% without it to 84.8% in this setting. Furthermore, we find that training a learned ACT head, even when the predicted halting signal is not used for dynamic early exit during training, can mitigate overfitting, as shown in Fig. 12 ( Training Head + No Early Halt vs. No Early Halt).

This hierarchy dependence is one reason we report ACT as part of the training-dynamics recipe rather than as a purely inference-side add-on.

Cost accounting explains why long-trajectory training is feasible.