Solve the Loop: Attractor Models for Language and Reasoning

arxiv:2605.12466

Original source · Paper JSON

Reasoning and the "physics" of language models · 2026-05-12

Cites (13) · Cited by (2)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4

Contents

• Solve the Loop: Attractor Models for Language and Reasoning
• Abstract
• 1 Introduction
• 1.1 Contributions
• 2 Background: Looped Architectures
• 3 Solve the Loop with Attractor Models
• 3.1 Attractor Model: Backbone and Attractor Modules
• 3.2 Training and Inference of Attractor Models
• 3.3 Equilibrium Internalization and Stability in Attractor Models
• 4 Experiments
• 4.1 Attractor Models Improve Large-Scale Language Modeling
• 4.2 Attractor Models Lead to Significant Gains in Hard Reasoning Tasks
• 4.3 Equilibrium Internalization and Test-Time Behavior
• 4.4 Ablations
• 5 Conclusion and Future Work
• Appendix A Related Work
• Appendix B Theory of Looped and Attractor Models
• B.1 Well-posedness and the implicit gradient
• Assumption 1 (Local contraction) .
• Theorem 1 (Well-posedness and implicit gradient) .
• Proof.
• B.2 Looped language models are fixed-point iterators
• Corollary 1 (LoopLM as a truncated approximation) .
• Proof.
• B.3 Where looped language models fall short
• B.3.1 Attractor Models
• B.3.2 Standard looped LMs
• B.3.3 Implicit-Gradient Barrier
• B.3.4 Interpretation
• Appendix C Hyperparamter and Experimental Settings
• Instructions for reporting errors