Hyperagents — part 1/7

Cleaned-text code points [0, 15736). Bibliography omitted here; source.md includes it.

\addtolist [1,2,5]Jeff Clune \authorformat \addtolist [4]New York University \affiliationformat

# HyperAgents

###### Abstract

Self-improving AI systems aim to reduce reliance on human engineering by learning to improve their own learning and problem-solving processes. Existing approaches to recursive self-improvement typically rely on fixed, handcrafted meta-level mechanisms, which fundamentally limit how fast such systems can improve. The Darwin Gödel Machine (DGM) ( Zhang et al., 2025b ) demonstrates that open-ended self-improvement is achievable in coding. Starting from a single coding agent, the DGM repeatedly generates and evaluates self-modified variants, forming a growing archive of stepping stones for future improvement. Because both evaluation and self-modification are coding tasks, gains in coding ability can translate into gains in self-improvement ability. However, this alignment does not generally hold beyond coding domains. We introduce hyperagents , self-referential agents that integrate a task agent (which solves the target task) and a meta agent (which modifies itself and the task agent) into a single editable program. Crucially, the meta-level modification procedure is itself editable, enabling metacognitive self-modification, improving not only task-solving behavior, but also the mechanism that generates future improvements. We instantiate this framework by extending DGM to create DGM-Hyperagents (DGM-H). By allowing the improvement procedure to evolve, the DGM-H eliminates the assumption of domain-specific alignment between task performance and self-modification skill, and can potentially support self-accelerating progress on any computable task. Across diverse domains (coding, paper review, robotics reward design, and Olympiad-level math-solution grading), the DGM-H improves performance over time and outperforms baselines without self-improvement or open-ended exploration, as well as prior self-improving systems like DGM. We further show that the DGM-H improves the process by which it generates new agents (e.g., persistent memory, performance tracking), and that these meta-level improvements transfer across domains and accumulate across runs. All experiments were conducted with safety precautions (e.g., sandboxing, human oversight). We discuss what safety entails in this setting and the broader implications of self-improving systems. DGM-Hyperagents offer a glimpse of open-ended AI systems that do not merely search for better solutions, but continually improve their search for how to improve.

## 1 Introduction

With appropriate safety considerations, AI systems that can improve themselves could transform scientific progress from a human-paced process into an autonomously accelerating one, thereby allowing society to realize the benefits of technological advances much earlier. Such self-improving AI seeks to continually improve its own learning and task-solving abilities. However, most existing self-improvement architectures rely on a fixed meta agent (i.e., a higher-level system that modifies a base system). This creates a limitation since the base system can only be improved within the boundaries defined by the meta agent’s design. Adding a meta-meta system to improve the meta agent does not solve this problem, it merely shifts the issue upward and ultimately leads to an infinite regress of meta-levels. To overcome this limitation and allow a system to modify any part of itself without being constrained by its initial implementation, the system must be self-referential, that is, able to analyze, modify, and evaluate itself ( Kirsch and Schmidhuber, 2022 ; Zhang et al., 2025b ) . When the mechanism of improvement is itself subject to improvement, progress can become self-accelerating and potentially unbounded ( Lu et al., 2023 ) .

The Darwin Gödel Machine (DGM) ( Zhang et al., 2025b ) demonstrates that open-ended self-improvement is achievable in coding. In the DGM, agents generate and evaluate modifications to their own code, and successful variants are retained in an archive as stepping stones for further improvement. However, the DGM relies on a handcrafted, fixed mechanism to produce self-improvement instructions ( Appendix B ). This mechanism analyzes past evaluation results and the agent’s current codebase to generate an instruction directing where the agent should self-improve. This mechanism is not modifiable. Hence, the DGM’s capacity for self-improvement is bottlenecked by this fixed instruction-generation step. Despite this handcrafted step, the DGM can still improve at self-improving. Because both evaluation and self-modification are coding tasks, improvements in evaluation performance directly reflects the agent’s capacity to generate effective self-modifications. To improve at self-improving, the DGM relies on a limiting assumption: that the skills required to solve the evaluation tasks are the same as those required for effective self-reflection and self-modification. This assumption is unlikely to hold outside coding domains, where task-solving skills may differ substantially from the skills needed to analyze failures, propose effective self-improvements, and implement them.

This work introduces hyperagents , self-referential agents that can in principle self-improve for any computable task. Here, an agent is any computable program, optionally including calls to foundation models (FMs), external tools, or learned components. A task agent solves a given task. A meta agent modifies agents and generates new ones. A hyperagent combines the task agent and the meta agent into a single self-referential, modifiable program, such that the mechanism responsible for generating improvements is itself subject to modification. As a result, a hyperagent can improve not only how it solves tasks (i.e., the task agent), but also how it generates and applies future modifications (i.e., the meta agent). Because its self-improvement mechanism is itself modifiable, we call this metacognitive self-modification . We extend the DGM with hyperagents, creating DGM-Hyperagents (DGM-H). The DGM-H retains the open-ended exploration structure of the DGM and extends the DGM with metacognitive self-modification. As with DGM, to support sustained progress and avoid premature convergence, the DGM-H grows an archive of hyperagents by branching from selected candidates, allowing them to self-modify, evaluating the resulting hyperagents, and adding them back to the archive. Because a hyperagent can modify its self-modification process, the DGM-H is not constrained by its initial implementation and can potentially self-improve for any computable task.

Across our experiments, the DGM-H demonstrates substantial and generalizable improvements in both task performance and self-improvement ability. On the Polyglot coding benchmark ( Gauthier, 2024 ) , the DGM-H achieves gains comparable to the most established prior self-improving algorithm ( Zhang et al., 2025b , the Darwin Gödel Machine,) , despite not being handcrafted for coding. Beyond coding, the DGM-H substantially improves performance on paper review ( Zhao et al., 2026 ) and robotics reward design ( Genesis, 2024 ) , with gains transferring to held-out test tasks and significantly outperforming prior self-improving algorithms, which struggle outside coding unless customized. Ablations without self-improvement or without open-ended exploration show little to no progress, highlighting the necessity of each component ( Section 5.1 ). Crucially, the DGM-H learns transferable mechanisms on how to self-improve (e.g., persistent memory, performance tracking) that systematically improve its ability to generate better task or meta agents over time. As a result, meta-level improvements learned by the DGM-H transfer across domains. Specifically, hyperagents optimized in one setting (i.e., paper review and robotics tasks) remain significantly effective at generating improved task agents in a different domain (i.e., Olympiad-level math grading) ( Section 5.2 ). We further show that self-improvements learned by the DGM-H in one setting can compound with continued self-improvement in another setting ( Section 5.3 ). This suggests that, given appropriate tasks, the DGM-H has the potential to achieve unbounded open-ended self-improvement over time. We discuss the safety implications of such open-ended self-improving systems and outline practical considerations for responsible deployment in Section 6 . Overall, hyperagents open up the possibility of improving their ability to improve while improving their ability to perform any computable task.

## 2 Related Work

Open-Endedness. Open-endedness refers to the ability of a system to continually invent new, interesting, and increasingly complex artifacts, extending its own frontier of discovery without a fixed objective or predefined end ( Stanley et al., 2017 ; Hughes et al., 2024 ) . Recent work has leveraged FMs as proxies for human interestingness and as versatile engines for generating and evaluating novel behaviors across diverse domains ( Zhang et al., 2024 ; Faldor et al., 2025 ) . Building on these advances, recent progress in open-ended learning ( Hu et al., 2025 ; Zoph and Le, 2017 ; Colas et al., 2023 ; Lehman et al., 2023 ) and quality-diversity algorithms ( Lehman and Stanley, 2011 ; Mouret and Clune, 2015 ; Bradley et al., 2023 ; Samvelyan et al., 2024 ; Ding et al., 2024 ; Pourcel et al., 2023 ; Coiffard et al., 2025 ; Dharna et al., 2025 ; Yuan et al., 2026 ) has shown that sustained exploration can produce diverse and increasingly capable artifacts across domains ranging from game-playing agents ( Klissarov et al., 2023 ; Klissarov et al., 2025 ; Wang et al., 2024 ) to scientific discovery ( Lu et al., 2024a ; Lu et al., 2024b ; Romera-Paredes et al., 2024 ; Novikov et al., 2025 ; Audran-Reiss et al., 2025 ) and robotic control ( Cully et al., 2015 ; Li et al., 2024 ; Grillotti et al., 2025 ) . Recent progress has shown that open-ended AI systems capable of continuously generating diverse and increasingly complex artifacts are possible ( Zhang et al., 2024 ; Faldor et al., 2025 ; Hu et al., 2025 ) . An important next step is to explore how such systems can achieve compounding improvement. In human scientific and technological progress, advances often build on prior advances not only by producing better artifacts, but also by improving the tools and processes that generate future discoveries, leading to accelerating innovation ( Good, 1966 ; Kwa et al., 2025 ) . Inspired by this pattern, we focus on open-ended systems that can improve not only the artifacts they generate, but also the mechanisms by which novelty and progress are produced ( Clune, 2019 ; Jiang et al., 2023 ) .

Self-improving AI. Early theoretical work on self-improving AI dates back to formal models of self-modifying agents ( Hutter, 2003 ) . One prominent example is the Gödel Machine ( Schmidhuber, 2003 ) , which proposes agents that rewrite themselves when provably beneficial, though such approaches remain impractical in real-world settings. Subsequent research explored self-improvement through adaptive neural systems, in which agents modify their own weights or learning dynamics via meta-learning ( Schmidhuber, 1993 ; Miconi et al., 2018 ; Javed and White, 2019 ; Beaulieu et al., 2020 ; Miconi et al., 2020 ; Irie et al., 2022 ; Chalvidal et al., 2022 ; Oh et al., 2025 ) , evolution ( Stanley and Miikkulainen, 2002 ; Lange et al., 2023 ; Qiu et al., 2025 ; Zhao et al., 2025 ) , or self-play ( Silver et al., 2016 ; Silver et al., 2017 ; Xia et al., 2025b ; Xia et al., 2026 ) . Notably, Silver et al. (2017) use self-play to iteratively improve neural network agents, achieving superhuman performance in domains such as Go and chess, although the underlying learning algorithms themselves remain fixed and human-designed. More recently, FMs have enabled self-improvement through iterative refinement of prompts ( Fernando et al., 2023 ; Wang et al., 2025a ; Zhang et al., 2025c ; Zhang et al., 2025a ; Ye et al., 2026 ) , reasoning traces ( Zelikman et al., 2022 ; Yin et al., 2025 ; Havrilla et al., 2024 ; Zhuge et al., 2024 ) , and entire code repositories ( Zhang et al., 2025b ; Wang et al., 2025b ; Xia et al., 2025a ) , as well as through systems that update model weights using self-generated data or interaction ( Wu et al., 2024 ; Zweiger et al., 2025 ; Wen et al., 2025 ; Wei et al., 2025b ) . Among these, the Darwin Gödel Machine (DGM) ( Zhang et al., 2025b ) stands out as a practical instantiation of recursive self-improvement in coding domains. However, despite their effectiveness, most existing approaches (including the DGM and its derivatives) rely on fixed, handcrafted meta-level mechanisms ( Appendix B ) that constrain how self-improvement can compound over time and generalize across domains.

Self-referential Meta-learning. Self-referential meta-learning studies systems that learn to improve the mechanisms by which learning occurs. Prior work has explored this idea in neural networks ( Kirsch and Schmidhuber, 2022 ; Jackson et al., 2024 ) and evolutionary methods ( Lu et al., 2023 ) . More recently, several works have explored self-referential improvement using FM-based agents ( Zelikman et al., 2024 ; Robeyns et al., 2025 ; Yin et al., 2025 ; Zhang et al., 2025b ) . The Darwin Gödel Machine (DGM) ( Zhang et al., 2025b ) and its successors ( Wang et al., 2025b ; Xia et al., 2025a ; Weng et al., 2026 ) instantiate recursive self-improvement through self-modification, primarily in coding domains. However, these approaches improve at improving primarily within coding tasks only. In the DGM and related systems, a coding agent is tasked with improving itself, and the resulting improved coding agent is then used in subsequent self-improvement steps to generate an even better version of itself. Because both the evaluation task and the self-modification process involve coding, improving the coding agent also enhances the system’s ability to carry out future self-improvements. However, this property only holds when the evaluation task and the self-modification task are closely aligned. For example, if the evaluation task were instead poetry writing, improving an agent’s poetry-writing ability would not necessarily improve its ability to modify its own code. Prior work therefore relies on an alignment between the evaluation task and the skills required for self-improvement. In contrast, hyperagents do not assume such alignment, because the self-modification mechanism is fully modifiable and not tied to any particular task domain. Hence, hyperagents can improve both task performance and the process of improvement itself across any computable task.

## 3 Methods

We introduce hyperagents, self-referential agents that unify task execution and agent generation into a single modifiable program. A hyperagent can improve not only how it solves tasks but also how it generates future improvements. To enable sustained and accumulating progress, we instantiate hyperagents by building directly on the Darwin Gödel Machine (DGM) to form DGM-Hyperagents (DGM-H). The DGM provides an open-ended, population-based exploration process that maintains an archive of progressively improving agents, allowing successful variants to serve as stepping stones for future gains. DGM-H retains this open-ended evolutionary structure and extends it by making the entire meta-level modification mechanism editable ( Figure 1 ). By allowing agents to modify not only how they solve tasks but also how they improve themselves, the DGM-H has the potential to open-endedly self-improve on any computable task.