Toward Training Superintelligent Software Agents through Self-Play SWE-RL

arxiv:2512.18552

Original source · Paper JSON

Agents, open-endedness, AGI · 2025-12-21

Cites (3) · Cited by (2)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4

Contents

• Toward Training Superintelligent Software Agents through Self-Play SWE-RL
• Abstract
• Keywords:
• 1 Introduction
• 2 Self-Play SWE-RL
• 2.1 Overview
• 2.2 Input Assumptions
• 2.3 Agentic Bug Injection
• Bug artifact.
• Complex bug generation.
• Consistency validation.
• Higher-order bugs.
• Reward design.
• 2.4 Agentic Bug Repair
• Construct the buggy codebase.
• Initial prompt.
• Bug-repair process.
• Reward design.
• 3 Evaluation
• 3.1 Experimental Setup
• Base model.
• Evaluation setup.
• 3.2 Baseline Comparison
• 3.3 Ablation of Self-Play Components
• 3.4 Ablation of Bug Injection Methods
• 3.5 Reward Ablation
• 4 Related Work
• Agent scaffolding for software engineering.
• Training software agents via reinforcement learning.
• Data and environments for training software agents.
• 5 Conclusion
• Acknowledgement
• Impact Statement
• Appendix A Analysis of the Challenger-Solver Game
• A.1 The Challenger Has an Optimal Target Solve Rate
• A.2 The Challenger Strategy
• The dominant challenger.
• The tunnel-vision challenger.
• Improving natural language communication with humans.
• Mitigations.
• Appendix B Opposing Incentives in the Reward Design
• Appendix C Experimental Details
• Training details.
• Evaluation noise.
• Appendix D Prompt Templates
• D.1 Bug Injection
• D.2 Bug Solving
• Appendix E Discussion
• E.1 Theoretical Analysis
• E.2 Limitations
• E.3 Unsuccessful Attempts
• E.4 Future Work
• Instructions for reporting errors