Recirculation

arxiv:2608.17981

Original source · Paper JSON

LLMs: architectures, context, training · 2026-08-18

Cites (8) · Cited by (1)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4
5. Part 5

Contents

• Recirculation
• Abstract
• 1 State tracking
• 2 Proposed method: Recirculation
• 3 Related research
• 4 Experiments
• 4.1 Hyperparameter sweeps
• 4.2 Perplexity evaluation
• 4.3 Normalization and ramping
• 4.4 Analysis of recirculation
• 4.4.1 Robustness across model families
• 4.4.2 Recirculation versus temperature tuning
• 4.4.3 Recirculation versus looping
• 4.4.4 Which tokens benefit from recirculation?
• 4.5 Generative tasks
• 4.5.1 Instruction following
• 4.5.2 Contextualization
• 4.5.3 Multiple-choice and single-token response tasks
• 4.5.4 GSM8k
• 4.6 Adaptive recirculation
• 5 Discussion
• 5.1 Model-design affordances
• 5.2 Limitations and future directions
• 6 Acknowledgements
• Appendix A Unrolled recirculation architecture
• Appendix B Recirculation implementation details
• B.1 Hyperparameter sweeps
• B.2 Perplexity evaluation
• B.3 Normalization and ramping
• Appendix C Basic results
• C.1 Robustness across architectures
• C.2 Recirculation versus temperature tuning
• C.3 Recirculation versus looping
• C.4 Which tokens benefit from recirculation?
• Appendix D Downstream evaluations
• D.1 Instruction following
• D.2 Contextualization
• D.3 Multiple-choice and single-token response tasks
• D.4 Standard benchmark datasets: GSM8k
• D.5 Adaptive recirculation
• Instructions for reporting errors