RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments
arxiv:2511.07317
Post-training · 2025-11-10
Complete cleaned Markdown · Source Markdown including bibliography
Read in parts
Contents
- RLVE : Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments
- Abstract
- Keywords:
- 1 Introduction
- 2 Methodology
- 2.1 Verifiable Environment
- 2.2 RL with Adaptive Verifiable Environments
- 2.3 RL Algorithm
- 3 RLVE-Gym : A Suite of 400 Environments Created through Environment Engineering
- 3.1 Environments as Pedagogical Tools
- 3.2 Verification via Environment Advantages
- 3.3 Designing Configurable Difficulty
- 4 Analyzing Components of RLVE
- 4.1 Adaptivity for Unstalled and Efficient Learning
- 4.2 Environment Scaling as a Key Driver of Generalizable Reasoning Capabilities
- 5 Scaling Up RL Training with RLVE
- 5.1 Scaling beyond Data Saturation
- 5.2 Compute-Efficient Scaling
- 6 Related Work
- RL training of LMs on procedurally generated data.
- Adaptive data for LM training.
- 7 Discussion on Future Work
- Scaling up RL with adaptive non-verifiable environments.
- Model-based automatic environment engineering.
- Acknowledgements
- Impact Statement
- Appendix A Details of RLVE
- Appendix B Details of RLVE-Gym
- B.1 Representative Sources of Verifiable Environments with Example Environments in RLVE-Gym
- Verifiable environments inspired by programming competition problems.
- Verifiable environments of mathematical operations.
- Verifiable environments of optimization problems.
- Verifiable environments of classical algorithmic problems.
- Verifiable environments of logical puzzles.
- Verifiable environments of NP-complete problems.
- B.2 Full List of 400 Verifiable Environments from RLVE-Gym
- Appendix C RL Training Details
- Appendix D Evaluation Details
- Appendix E Details of Training Environment Collection
- Instructions for reporting errors