Skip a Layer or Loop It? Learning Program-of-Layers in LLMs

arxiv:2606.06574

Original source · Paper JSON

LLMs: architectures, context, training · 2026-06-04

Cites (2) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4

Contents

• Skip a Layer or Loop It? Learning Program-of-Layers in LLMs
• Abstract
• Keywords:
• 1 Introduction
• 2 Dynamic Inference as a Program-of-Layers ( PoLar ) in Large Language Models
• 3 Learning Program-of-Layers ( PoLar ) in Large Language Models
• 3.1 Program Representation
• 3.2 Program-of-Layers ( PoLar ) Prediction Network
• 4 Experiments
• 4.1 Experimental Setup
• 4.2 Main Results
• 4.3 Out-of-Distribution Performance
• 5 Related Works
• 6 Conclusion
• Impact Statement
• Appendix A Related Work
• Layer Pruning and Early-Exit Neural Networks
• Looped Transformer and Recurrent Depth
• Dynamic Routing and Modular Inference
• Appendix B Searching the Execution Program Space
• B.1 Execution Program Space
• B.2 Search Space Constraints
• B.3 Monte Carlo Tree Search Formulation
• State and Actions.
• Reward.
• Tree Policy.
• B.4 Search Algorithm
• Appendix C Experimental Results
• C.1 In-Distribution Performance
• C.2 Out-of-Distribution Generalization
• Appendix D Empirical Details
• D.1 Dataset Details
• Revision note.
• D.2 Training Configuration
• Optimization.
• D.3 Predictor Size
• D.4 Direct Prompting
• Instructions for reporting errors