AI Finds A Way — part 1/13

Cleaned-text code points [0, 15654). Bibliography omitted here; source.md includes it.

# AI Finds A Way

###### Abstract

Artificial Intelligence (AI) algorithms frequently learn creative and unexpected solutions, surprising even expert researchers who develop and study them. They often astonish practitioners by discovering unanticipated behavior, exploiting loopholes in reward signals, or spontaneously uncovering previously unknown scientific phenomena. However, accounts of such unconventional behavior across machine learning are seldom formally documented; instead, they circulate primarily as informal cautionary tales or amusing anecdotes amongst AI researchers. This work presents 26 curated firsthand anecdotes from various machine learning subfields representing the work of over 100 researchers. It includes algorithms devising seemingly impossible quantum optics experiments, bypassing human oversight on physical manipulation tasks, seeking out in-game drug-induced hallucinations to feign success in video games, and more. These anecdotes showcase the capability of modern AI systems to circumvent human-imposed design limitations and discover unexpected solutions to the tasks we train them on. Furthermore, these accounts are particularly important for the safety of future AI systems. They illustrate the fundamental challenge of aligning models with human values without diminishing their creativity, so they can make surprising discoveries without producing surprising, potentially harmful outcomes. The paper first details instances of AI achieving superhuman success through reinforcement learning across many challenging domains. However, we show how reward-driven optimization can fail when the model learns to hack an underspecified reward or unarticulated constraint. We then present case studies suggesting that harnessing internet-scale foundation models (FMs) has not resolved these fundamental challenges and, in fact, can supercharge existing failure modes. Nevertheless, we argue that when channeled through well-scoped objectives, rigorous verification, and human-in-the-loop scientific judgment, these same learning dynamics can be harnessed to accelerate scientific discovery. Finally, we hope this work provides a consolidated resource to inform future research and demonstrates that the tendency toward unexpected behaviors is commonplace in modern AI, highlighting the need to anticipate and manage AI’s capacity for innovative, yet unpredictable, solutions.

## 1 Introduction

As the character Dr. Malcolm warns in Jurassic Park, “Life finds a way.” We can extrapolate this lesson to artificial intelligence (AI): AI too finds a way. Despite our attempts to try to control AI learning algorithms, they often find a way to circumvent our constraints or game the system ( Lehman et al., 2020 ) . Even when we ask them to be creative and make innovations, AI systems frequently discover solutions that astonish even the researchers who develop and study them ( Lehman et al., 2020 ) . For example, deep reinforcement learning ( Mnih et al., 2015 ; Sutton and Barto, 2018 ) agents routinely uncover exploits and loopholes in reward functions ( Amodei et al., 2016 ) ; large language models ( OpenAI, 2022 ; Anthropic, 2024 ; Comanici et al., 2025 ) can manipulate labeling systems to maximize perceived performance ( Pourcel et al., 2024 ) ; and automated scientific discovery algorithms have proposed experiments initially dismissed as impossible but later validated ( Krenn et al., 2017 ) .

Despite the prevalence of surprising discoveries across various machine learning (ML) subfields—including but not limited to deep learning ( Goodfellow et al., 2016 ) , reinforcement learning ( Sutton and Barto, 2018 ) , natural language processing ( Jurafsky and Martin, 2026 ) , evolutionary computation ( Lehman et al., 2020 ) , and AI for science ( Wang et al., 2023a ) —these and similar anecdotes often remain informally circulated rather than formally documented as case studies. Such accounts in the artificial life and evolutionary computation fields, where artificial evolutionary algorithms routinely upended researcher expectations or revealed unanticipated system constraints, were primarily shared informally before being collected by Lehman et al. (2020) . We build on Lehman et al. (2020) by collecting important ML anecdotes in one place, including capturing important anecdotes from papers, interviews, blog posts, and gathering new anecdotes and/or details about them. Ultimately, this work serves as a sequel to Lehman et al. (2020) , expanding its scope to additional subfields of AI and highlighting their underlying lessons for the broader research community while verifying the historical accuracy of the anecdotes contained herein. Through documenting and analyzing these unexpected outcomes, this paper provides a foundational resource for researchers and offers a more rigorous understanding of AI’s creative and exploitative potentials.

We illuminate the depth of creativity that modern AI can exhibit while highlighting potential risks and unintended consequences in its deployment. These accounts hold significant implications beyond mere novelty. If AI can circumvent guardrails in creative ways, we cannot guarantee its safe deployment. Recognizing that surprising creativity is not merely an isolated artifact of evolutionary computation—as focused on by Lehman et al. (2020) —but rather a broader characteristic of complex learning-based systems positions us to better anticipate, understand, and guide the development of AI technologies.

The next sections present 26 curated anecdotes representing the work of over 100 researchers. Most of the accounts (16/26) collected here are newly documented and recounted firsthand by the scientists themselves. The remainder were harvested from public descriptions by the authors (e.g., public interviews or scientific publications); in some cases, we communicated with the authors to clarify or obtain new details. Direct quotes from the researchers are placed in block quotes to make clear which words are theirs. Unless otherwise stated, block quotes come from text the authors sent us or are quotes from public interviews or publications (for public material, we provide a citation to the source). SI Section 8 lists the source of each anecdote. Additionally, we have established a repository for these anecdotes and future ones. We invite readers to submit future anecdotes you may have to www.github.com/aadharna/aifw .

We do not attempt to catalogue every publicized example of alarming AI behavior or capabilities. While we do provide one such example in Section 4.2.6 , in general, highly staged and/or scaffolded demonstrations in which an AI system is given a test designed to evaluate whether it will resort to manipulative or coercive behavior, such as threatening or blackmailing a person, are mostly outside the scope of this collection. For example, such behavior includes instances where a model engaged in insider trading and attempted subsequent cover-ups when placed in a simulated corporate environment and instructed to act like a stock trader ( Scheurer et al., 2024 ) . These behaviors are being actively studied ( Hubinger et al., 2024 ; Van Der Weij et al., 2025 ; Lynch et al., 2025 ; Greenblatt et al., 2024 ; Meinke et al., 2025 ; Hopman et al., 2026 ) , and such cases matter for broader AI safety discussions ( Bengio et al., 2024 ) . Writ large, however, we focus on anecdotes where surprising behaviors emerged while pursuing research objectives unrelated to evaluating those specific behaviors.

In reviewing the anecdotes, we clustered them into five somewhat overlapping categories. The work is thus structured around the following core observations: AI that learns to interact with the world, in particular via reinforcement learning but also search more generally, can create new knowledge, discoveries, and improvements above and beyond current human knowledge ( Section 3 ). However, single-mindedly optimizing objectives can have unexpected effects. AI can game a training signal and learn a solution that satisfies the letter of the task, but does not solve the task as desired ( Section 4.1 ). Additionally, the model can learn to break the intended constraints of its training environment, leading to behaviors that developers thought were impossible ( Section 4.2 ). As a result, ML methods often give you what you asked for but not what you wanted, and find solutions that violate the spirit of the task in surprising ways. Furthermore, in the modern era of powerful generative AI models, these aforementioned failure modes do not disappear. In fact, these pathologies can be supercharged by the emergent capabilities of modern AI models ( Section 5 ). While optimization can exploit imprecise objectives or constraints in risky ways, this same tendency can also be harnessed for discovery. By proposing surprising but empirically testable hypotheses, mechanisms, and designs, researchers can filter these unexpected outputs into credible candidates for experimentation, ultimately utilizing AI’s propensity to be creative to accelerate scientific progress ( Section 6 ). Section 7 highlights shared threads from the prior chapters, connects these lessons to broader research directions in AI, and discusses the potential ramifications of deploying AI technology with these capabilities.

## 2 Background

Many algorithms in the field of AI operate by identifying complex patterns and correlations within vast datasets. The primary class of machine learning algorithms focused on by the community as a whole, as well as in this manuscript, is deep learning ( Goodfellow et al., 2016 ; Bishop, 2006 ; Murphy, 2022 ) . Deep learning utilizes artificial neural networks which are computational structures loosely inspired by the human brain and characterized by multiple layers of interconnected processing units. These networks excel at learning hierarchical representations directly from raw data ( Yosinski et al., 2014 ; Goodfellow et al., 2016 ) . Given sufficient data (e.g., millions of images, vast amounts of text, or extensive game logs), deep learning models can automatically discover intricate patterns necessary for tasks like classifying images with high accuracy ( Krizhevsky et al., 2012 ; Simonyan and Zisserman, 2015 ; He et al., 2016 ) , understanding and generating human language ( Brown et al., 2020 ; Devlin et al., 2019 ; Vaswani et al., 2017 ) , or recognizing speech ( Hinton et al., 2012 ; Graves and Jaitly, 2014 ) . Much of the success of deep learning in these fields comes from the expansive nature of the datasets used to train the models, such as training on all data available on the internet.

While most original, canonical deep learning models were trained with supervised learning (passively mapping inputs to outputs), many of the anecdotes we collect here arise from interactive algorithms that learn through trial and error, of which reinforcement learning is a prime example. Reinforcement learning involves an agent learning to maximize a numerical reward signal within an environment via repeated trial-and-error experimentation on the task ( Sutton and Barto, 2018 ) . For example, a home cleaning robot might learn to put away a mug by receiving positive rewards for actions that bring it closer to a goal, such as placing it in a sink, and negative rewards for failures like breaking the mug. Whether the goal is to clean the house, win in chess (getting a score of +1) or to get a high score in Atari ( Mnih et al., 2015 ) games, the agent’s objective is to learn a policy that maximizes long-term cumulative reward. Agents trained in this way have been at the heart of many examples of superhuman performance in board and video games, as measured against top professionals ( Silver et al., 2017 ; Vinyals et al., 2019 ; OpenAI et al., 2019 ; Bakhtin et al., 2022b ; Brown and Sandholm, 2017 ; Wurman et al., 2022 ) . Deep reinforcement learning (DRL) served as the engine for these successes, combining reinforcement learning’s trial-and-error learning and sequential decision-making with deep learning’s pattern recognition to associate high reward with certain actions and/or states observed by the model. More generally, DRL has enabled agents to master environments ranging from robotic control ( Levine et al., 2016 ) to atmospheric balloon navigation ( Schuler et al., 2025 ; Bellemare et al., 2020 ) . As we will see in Section 3 , these successes showcase how RL can master complex strategic thinking, long-term planning, and even natural-language negotiation.

Recently, the field has seen the development of powerful foundation models (FMs) that are trained via supervised learning on vast amounts of internet data, then taught specialized competencies like math and coding with reinforcement learning ( Wen et al., 2025 ; DeepSeek-AI, 2024 ; Su et al., 2025 ; Chen et al., 2021 ) . The scale of these models and breadth of training data have led to the emergence of novel behaviors and capabilities that were not explicitly programmed or anticipated ( Comanici et al., 2025 ; OpenAI, 2022 ; Anthropic, 2024 ; Grattafiori et al., 2024 ) . Furthermore, FMs can synthesize and generate new information from their training data, allowing them to solve problems in surprising ways ( Brown et al., 2020 ; Perez et al., 2023 ) . However, this development has not been without its share of unexpected outcomes, and the very complexity that gives FMs their power can also lead to unforeseen failure modes.

Models trained with RL are exceptionally prone to discovering behaviors that hack the objective and break developer-imposed constraints ( Wang et al., 2023b ; Amodei et al., 2016 ; Krakovna et al., 2020 ) . These surprising behaviors often arise from the tension between a human designer’s implicit goals (e.g., clean the house and place everything where it belongs) and an agent’s explicit objective of maximizing its reward function (e.g., minimize visible clutter—which could lead the robot to hide items in the wrong cabinets and under tables). Many of the anecdotes we present (starting in Section 4.1 ) arise where powerful learners are given objectives that only imperfectly capture human intent or are improperly specified. While such misalignment might be merely inconvenient in a domestic setting, the implications become alarming as the scope of deployment widens, including in safety- and mission-critical applications such as fusion reactor control ( Degrave et al., 2022 ) , as will be discussed in Section 6 . For now, though, we begin our tour of surprising behavior in perhaps the most familiar setting for AI: games, where AI systems have long surpassed the best humans.

## 3 Superhuman and Optimal Performance

Reinforcement learning (RL) and search-based algorithms have surpassed the highest levels of human performance in games like Go ( Silver et al., 2016 ) and chess ( Campbell et al., 2002 ; Silver et al., 2018 ) . The true surprise of this era, however, was not that the algorithms solved many of the tasks they were trained on, but the specific nature of their victories: they discarded established human strategies ( Brown and Sandholm, 2017 ; Silver et al., 2018 ; Wurman et al., 2022 ) . Furthermore, they yielded novel ideas that have changed how expert communities approach these games ( Pascutto, 2019 ; Shin et al., 2023 ) . Algorithmic discovery of genuine novelty is the thread running through the anecdotes that follow: machines began not just to dominate domains once thought to be the exclusive province of human reasoning, but to enrich those domains with ideas no human had produced.