Less is More: Recursive Reasoning with Tiny Networks

arxiv:2510.04871

Original source · Paper JSON

Reasoning and the "physics" of language models · 2025-10-06

Cites (3) · Cited by (12)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3

Contents

• Less is More: Recursive Reasoning with Tiny Networks
• Abstract
• Keywords:
• 1 Introduction
• 2 Background
• 2.1 Structure and goal
• 2.2 Recursion at two different frequencies
• 2.3 Fixed-point recursion with 1-step gradient approximation
• 2.4 Deep supervision
• 2.5 Adaptive computational time (ACT)
• 2.6 Deep supervision and 1-step gradient approximations replaces BPTT
• 2.7 Summary of HRM
• 3 Target for improvements in Hierarchical Reasoning Models
• 3.1 Implicit Function Theorem (IFT) with 1-step gradient approximation
• 3.2 Twice the forward passes with Adaptive computational time (ACT)
• 3.3 Hierarchical interpretation based on complex biological arguments
• 4 Tiny Recursion Models
• 4.1 No fixed-point theorem required
• 4.2 Simpler reinterpretation of z H z_{H} and z L z_{L}
• 4.3 Single network
• 4.4 Less is more
• 4.5 attention-free architecture for tasks with small fixed context length
• 4.6 No additional forward pass needed with ACT
• 4.7 Exponential Moving Average (EMA)
• 4.8 Optimal the number of recursions
• 5 Results
• 6 Conclusion
• Acknowledgements