Reinforcement Pre-Training

arxiv:2506.08007

Original source · Paper JSON

Data, training, optimization · 2025-06-09

Cites (4) · Cited by (3)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2

Contents

• Reinforcement Pre-Training
• Abstract
• 1 Introduction
• 2 Preliminary
• Next-Token Prediction (NTP)
• Reinforcement Learning with Verifiable Rewards (RLVR)
• 3 Reinforcement Pre-Training
• 3.1 Pre-Training Task: Next-Token Reasoning
• 3.2 Pre-Training with Reinforcement Learning
• 3.3 Pre-Training Setup
• 3.4 Evaluation of Pretrained Models
• Language Modeling
• Reinforcement Fine-Tuning on Downstream Tasks
• 4 Experiments
• 4.1 Language Modeling
• 4.2 Scaling Properties of Reinforcement Pre-Training
• 4.3 Reinforcement Fine-Tuning with RPT
• 4.4 Zero-Shot Performance on End Tasks
• 4.5 Next-Token Reasoning Pattern Analysis
• 5 Related Work
• Scaling Paradigms of Large Language Models
• Reinforcement Learning for Large Language Models
• 6 Conclusion and Future Work
• Acknowledgement
• Appendix A Design Choices of Reward
• Appendix B Hyperparameters Used for Reinforcement Pre-Training
• Appendix C Detailed Results on End Tasks
• Appendix D Impact of Prompt Templates
• Appendix E Keywords for Reasoning Pattern Analysis
• Appendix F Case Studies
• Instructions for reporting errors