Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning

arxiv:2605.21488

Original source · Paper JSON

Reasoning and the "physics" of language models · 2026-05-20

Cites (10) · Cited by (3)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4
5. Part 5
6. Part 6
7. Part 7

Contents

• Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning
• Abstract
• Keywords:
• 1 Introduction
• 2 Background and Problem Formulation
• 3 From Feedforward Predictors to Iterative Reasoners
• Weight-tied structure.
• Truncated gradients.
• Hierarchical iterations.
• Supervision placement and optimization schedule.
• Adaptive computation time (ACT).
• Overview.
• 4 Iterative Models as Attractor Dynamics
• 4.1 From Fixed-Point Convergence to Attractors
• 4.2 Landscape Modes and Scaling Implications
• 5 Shaping Attractor Landscapes
• 5.1 Randomized State Initialization
• 5.2 Path Stochasticity via Noise Injection
• 6 Experiments
• Task representation.
• Evaluation metrics.
• Baselines.
• 6.1 From Feedforward Models to Iterative Models
• Summary and takeaway.
• 6.2 From Iterative Models to Equilibrium Reasoners
• Summary.
• 6.3 Adaptive Computation for Budget-Elastic Inference
• 7 Related Work
• Iterative weight-tied models.
• Compute allocation in weight-tied models.
• 8 Conclusion
• Appendix A Additional Results, Analyses, and Findings
• A.1 Attractor Formulation and Residual Diagnostics
• Fixed-point residual as a local convergence diagnostic.
• Implicit gradient conditioning.
• A.2 From Feedforward Models to Iterative Models: Training-Dynamics Ablations and Diagnostics
• A.3 Additional Experiments on Randomized State Initialization
• Learnable initial state.
• Empirical observation.
• Scope and limitations.
• Randomness scale.
• A.4 Additional Experiments on Path Stochasticity
• A.5 Generalization Beyond the Main Setting
• A.6 Seed Stability Diagnostics
• Appendix B Qualitative Study
• Appendix C Dataset Details and Task Definitions
• C.1 Dataset and Benchmark Specifications
• Sudoku-Extreme
• Naming convention.
• C.2 Datasets and Task Definitions Shape Attractor Landscapes
• Appendix D Method and Experimental Details
• D.1 Architecture and Hyperparameters
• D.2 Learning-Rate Control for Feedforward Baselines
• D.3 Evaluation Metrics
• Averaged exact acc.
• Top-1 convergence accuracy.
• Majority vote accuracy.
• Path independence across restarts.
• D.4 Compute Accounting for Iterative Inference
• Appendix E Extended Related Work and Discussion
• Deep Equilibrium Models.
• Latent reasoning models.
• Theoretical and mechanistic analyses of weight-tied iteration.
• Flat minima.