SPADE: Self-Play in Adaptive Synthetic Executable Environments

arxiv:2608.19197

Original source · Paper JSON

Agents, open-endedness, AGI · 2026-08-19

Cites (11) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4
5. Part 5
6. Part 6
7. Part 7
8. Part 8

Contents

• SPADE ♠ \spadesuit : Self-Play in Adaptive Synthetic Executable Environments
• Abstract
• 1 Introduction
• 2 Related Work
• 3 Preliminaries
• 3.1 Markov Decision Processes and Gym-Style Environments
• 3.2 RLVR and GRPO
• 4 SPADE : Self-Play in Adaptive Synthetic Executable Environments
• 4.1 Dual-Role Self-Play and Code-as-Environment
• 4.2 Hint-Based Regret Reward
• 4.3 Environment Design
• 5 Experimental Setup
• 5.1 SPADE Training Recipe
• 5.2 Domain: Games
• 5.3 Domain: Tool Use
• 6 Experimental Results
• 6.1 Quantitative Analysis
• 6.2 Qualitative Analysis
• 7 Ablations
• 7.1 Ablation: Environment Designer Adaptation
• 7.2 Ablation: Environment Designer Reward
• 8 Scaling Results
• 9 Discussion
• 10 Conclusion
• Acknowledgments
• Appendix E Theoretical Analysis [back to contents]
• E.1 Setup and Assumptions
• Environment class.
• Distribution notation.
• Inference protocol.
• Fine-tuning budget.
• Returns with and without hints.
• Game payoffs.
• Pure Nash equilibrium.
• Optimal values.
• Assumption E.1 (Sound generation) .
• Assumption E.2 (Articulated hints) .
• Assumption E.3 (Internalizability of hinted behavior) .
• E.2 Main Results
• Lemma E.4 (Hint-regret equals hint-free regret) .
• Proof.
• Theorem E.5 (Nash equilibria imply hint-free optimality on every environment) .
• Proof.
• Appendix F Method and Experiment Details [back to contents]
• F.1 System Prompts and Templates
• Environment-generation prompt.
• Multi-turn environment-generation prompt.
• Corpus-grounded interactive generation prompt.
• Tool-use generation prompt.
• Reasoning Agent gameplay prompt.
• F.2 Implementation Details
• F.3 Reproducibility
• F.3.1 Comparability Notes for Table 2
• Appendix G Extended Ablations [back to contents]
• Appendix H Extended Related Work [back to contents]
• H.1 Self-Play for LLMs
• H.2 Unsupervised Environment Design and Open-Endedness
• H.3 Synthetic Environment Generation
• H.4 Environment Scaling
• H.5 Agentic Memory Design and Self-Improving Code Systems
• Appendix I Extended Quantitative Analysis [back to contents]
• I.1 Games
• I.1.1 Environment Quality
• I.1.2 Environment Diversity
• Metric and calibration.
• Diversity is maintained at the ceiling.
• Corpus grounding drives diversity.
• Scope and limitations.
• Appendix J Extended Qualitative Analysis [back to contents]
• J.1 Games
• J.1.1 Additional Privileged-Hint Examples
• J.1.2 Environment Evolution over Training
• J.1.3 Matched-Step Generation Contrast
• J.2 Tool Use
• J.3 Generated Environment Gallery
• J.3.1 A Minimal Generated Environment
• J.3.2 Full Source of Two Exemplar Generated Environments
• Training step 0: CarOwnershipDisputeEnv (370 lines).
• Training step 384: ThermodynamicCycleManipulationLabEnv (376 lines).
• J.3.3 Full Source of a Generated Tool-Use Environment
• Training step 392: CustomerSupportTicketWorkflowEnv (200 lines).
• Instructions for reporting errors