AI Finds A Way

arxiv:2608.23875

Original source · Paper JSON

Agents, open-endedness, AGI · 2026-08-26

Cites (13) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4
5. Part 5
6. Part 6
7. Part 7
8. Part 8
9. Part 9
10. Part 10
11. Part 11
12. Part 12
13. Part 13

Contents

• AI Finds A Way
• Abstract
• 1 Introduction
• 2 Background
• 3 Superhuman and Optimal Performance
• 3.1 AlphaGo: Move 37 and Creativity from AI Methods ( Silver et al., 2016 ; Fridman, 2020 )
• 3.2 Libratus Surprises the Poker Community with Overbets ( Brown and Sandholm, 2017 ; Imbue, 2023 )
• 3.3 Professional Diplomacy Players Meet Diplodocus and Cicero ( Bakhtin et al., 2022b ; Bakhtin et al., 2022a )
• 3.4 Opponent Shaping Leads to the Spontaneous Rediscovery of Game-Theoretic Strategies ( Lu et al., 2022 ; Foerster et al., 2018 )
• 3.5 Takeaways
• 4 Reward Hacking
• 4.1 Exploiting Reward and Score Functions
• 4.1.1 Playing CoastRunners Forever Rather than Winning the Game ( Amodei et al., 2016 ; Clark and Amodei, 2016 )
• 4.1.2 StarCraft II Agents Exploit Shield Regeneration ( Samvelyan et al., 2019 )
• 4.1.3 Exploiting Human Perception ( Christiano et al., 2017 ; Amodei et al., 2017 )
• 4.1.4 PAIRED Agents Learned to Collude ( Dennis et al., 2020 )
• 4.1.5 CycleGAN Learns to “Hide” Information ( Chu et al., 2017 )
• 4.1.6 RL Agent Farmed Flowers Instead of Catching ’Em All ( Whidden, 2024 )
• 4.1.7 Takeaways
• 4.2 Exploiting Environmental Weaknesses
• 4.2.1 Hide and Seek Playing Agents Break the Simulator ( Baker et al., 2020 )
• 4.2.2 Robotic Humanoid Walks Without Using Its Feet ( Batra et al., 2024 )
• 4.2.3 Playing the Model, Not the Game ( Ha and Schmidhuber, 2018 )
• 4.2.4 StarCraft II Agents Outsource Combat ( Samvelyan et al., 2019 )
• 4.2.5 The Evolved Radio ( Bird and Layzell, 2002 )
• 4.2.6 GPT-4 Recruits Humans to Circumvent CAPTCHA ( OpenAI et al., 2024 )
• 4.2.7 Takeaways
• 5 Foundation Models and Large Language Models: General-Purpose Intelligence
• 5.1 From Human Data to Phantom Crafting: VPT’s Shortcut to Failure ( Baker et al., 2022 )
• 5.2 Agent Takes Drugs to Reward Hack by Hallucinating Reaching the Goal ( Klissarov et al., 2024 )
• 5.3 Code Generator Deceives LLM Labeler ( Pourcel et al., 2024 )
• 5.4 Automated Vulnerability Probing System Exploits Vulnerability in Its Own Evaluator ( Samvelyan et al., 2024 )
• 5.5 Inescapable Wedding Parties
• 5.6 Claude 3 Realizes It Is Being Tested ( Albert, 2024 )
• 5.7 Takeaways
• 6 AI for Science
• 6.1 Magnetic Control of Tokamak Plasmas through Deep Reinforcement Learning ( Degrave et al., 2022 ; Chauhan, 2023 )
• 6.2 FunSearch: Mathematical Discoveries from Program Search with Large Language Models ( Romera-Paredes et al., 2023 )
• 6.3 The AI Scientist Breaks Out of Constraints ( Lu et al., 2024 )
• 6.4 Discovering a Quantum Result Thought to be Impossible, with Highly Productive Consequences ( Krenn et al., 2017 )
• 6.5 Takeaways
• 7 Discussion and Conclusion
• 7.1 Optimization Finds the Unexpected
• 7.2 When Oversight Becomes an Optimization Target
• 7.3 Beyond Better Objectives
• Acknowledgments
• Supplementary Material
• Table of Contents
• 8 Breakdown of Story Attribution
• 8.1 Call for Anecdotes
• 8.2 List of Anecdotes
• Instructions for reporting errors