Hyperagents — part 5/7
Cleaned-text code points [61860, 77547). Bibliography omitted here; source.md includes it.
In the Polyglot coding benchmark [ Gauthier, 2024 ] , each task consists of a software repository and a natural language instruction describing a desired change to the codebase. The agent is given access to the full repository and must modify the files to correctly implement the instruction, producing a patch (i.e., a set of code edits) applied to the repository. Performance is evaluated by running a predefined test suite on the modified repository. A task is considered to be successfully done if all tests pass. We follow the setup used in the DGM [ Zhang et al., 2025b ] , which largely mirrors the Polyglot leaderboard configuration, with one key difference: the leaderboard reports pass@2, allowing the agent to view feedback from ground-truth tests once, whereas we report pass@1, in which the agent never sees ground-truth test results. We adopt the same training and test splits as in the DGM. Training tasks are selected as a random subset of the full benchmark, comprising a total of 60 tasks. If an agent achieves more than 40% success on an initial 10-task subset, it is subsequently evaluated on the remaining 50 training tasks. There is no validation subset for this domain. As a final evaluation to more accurately assess performance improvements, we evaluate the generated agents on the full Polyglot benchmark, which consists of 165 unseen tasks.
Initial 10 training tasks for preliminary evaluation:
• go__dominoes
• cpp__all-your-base
• python__dominoes
• java__sgf-parsing
• javascript__robot-name
• rust__variable-length-quantity
• python__beer-song
• go__book-store
• javascript__bottle-song
• rust__bowling
Additional 50 training tasks for full evaluation:
• javascript__queen-attack
• rust__wordy
• python__dot-dsl
• java__satellite
• cpp__diamond
• rust__accumulate
• go__error-handling
• cpp__queen-attack
• rust__poker
• python__sgf-parsing
• rust__react
• java__ledger
• go__connect
• rust__macros
• javascript__triangle
• java__zipper
• java__bowling
• python__tree-building
• javascript__say
• java__wordy
• python__food-chain
• javascript__wordy
• python__poker
• javascript__grade-school
• cpp__gigasecond
• java__forth
• python__dominoes
• go__word-search
• javascript__simple-linked-list
• go__counter
• java__react
• javascript__ocr-numbers
• python__scale-generator
• java__go-counting
• rust__doubly-linked-list
• python__grade-school
• javascript__forth
• python__wordy
• java__mazy-mice
• cpp__bank-account
• python__zipper
• java__custom-set
• java__rest-api
• go__transpose
• rust__gigasecond
• rust__say
• go__food-chain
• rust__pig-latin
• go__markdown
• go__crypto-square
### C.2 Paper Review
The data in this domain are drawn from Zhao et al. [2026] . Each task in the paper review domain consists of the full text of an AI research paper. The agent must predict a binary accept or reject decision, simulating the role of a conference reviewer. Ground-truth labels correspond to real acceptance decisions from top-tier machine learning conferences, including ICLR 2024/2025 and NeurIPS 2023/2024. Performance is measured by classification accuracy with respect to these labels. We randomly sample tasks to construct training, validation, and test splits, each containing 100 tasks. During training, the agent is first evaluated on a subset of 10 tasks from the training split. If the agent succeeds on at least one of these tasks, it is then evaluated on the full set of 100 training tasks.
AI-Scientist-v2 [ Yamada et al., 2025 ] employs an AI reviewer to automatically improve generated AI research papers. We adopt the AI reviewer proposed in that work as our representative static baseline: ⬇
### C.3 Robotics Reward Design
Each task in the robotics reward design domain specifies a robotic control objective in the Genesis simulator [ Genesis, 2024 ] using a Go2 quadruped robot. The agent is given a textual description of the task (e.g., walk forward at a target velocity) and outputs a Python reward function, which is then used to train a RL policy [ Schulman et al., 2017 , i.e., PPO,] for the robot. Performance is evaluated by executing the trained RL policy in the simulator and computing the task performance measure (e.g., velocity tracking error). Scores are averaged over repeated evaluations to reduce variance due to stochasticity in reward generation or RL.
The training task requires generating a reward function that enables the robot to walk forward while tracking a target linear velocity. Performance is measured using the mean squared error between the commanded and actual walking velocities. During training, each agent is initially evaluated 3 repeated times on the same task, generating one reward function per evaluation. If at least one generated reward function yields a non-zero performance score, the agent is evaluated 3 additional times. The final performance score is reported as the average across the 6 evaluations. No separate validation task is curated for this domain.
To assess whether the same agent can generate suitable reward functions across different robotics tasks, we pair a relatively simple training task with a more challenging test task on the same robot. The test task requires generating a reward function that trains the robot to maximize torso height. Reward functions that are effective for forward walking do not induce jumping behaviors, which are more optimal for maximizing torso height. Moreover, directly incentivizing torso height (the performance measure) typically leads to a suboptimal standing behavior of standing stall. Achieving high performance therefore requires non-myopic reward design that encourages intermediate behaviors, such as lowering the torso before jumping.
The default reward function for the test task directly rewards the performance measure of maximizing torso height. This always produces a behavior in which the robot simply stands as tall as possible ( Figure 6 ): ⬇
### C.4 Olympiad-level Math Grading
In the Olympiad-level math grading domain, tasks are drawn from IMO-GradingBench [ Luong et al., 2025 ] . Each task consists of an Olympiad-level math problem, a candidate solution, reference solutions, and grading guidelines. The agent is required to assign a discrete score from the set {0, 1, 6, 7}, corresponding to the categories {incorrect, partial, almost, correct}. The agent’s output is a single numeric grade. Performance is measured by accuracy with respect to expert human annotations, with additional analyses provided in Section E.4 . We use randomly sampled subsets of tasks for training, validation, and testing, with 100 tasks in each split. During training, an agent is first evaluated on a subset of 10 tasks (out of 100). If the agent succeeds on at least one of these tasks, it is then evaluated on the full training set.
Representative static baseline (ProofAutoGrader) from Luong et al. [2025] : ⬇
## Appendix D Experiment Details
This appendix provides additional experimental details to support reproducibility of the results. We first summarize the FMs and hyperparameters used for self-modification and task evaluation across domains ( Section D.1 ), followed by an estimate of the computational cost of running the DGM-H in each setting ( Section D.2 ). We then formally define the improvement@k metric used to quantify an agent’s ability to produce improved variants under a fixed budget ( Section D.3 ). Finally, we describe the procedure used to select transfer agents for cross-domain experiments ( Section D.4 ).
### D.1 Hyperparameters for FMs
Table 3 summarizes the foundation models (FMs) used across experimental settings. For the Polyglot coding domain, we adopt the same FMs and temperature configurations as Zhang et al. [2025b] to ensure a fair comparison. In all other domains, we use Claude-4.5-Sonnet for self-modification, given its strong performance on coding. For task evaluation, we select the FM based on practical considerations, including computational cost, rate limits, response latency, and overall task competence. In the robotics reward design setting, where the agent must implement reward functions in code, we again use Claude-4.5-Sonnet. For Olympiad-level mathematics grading, which requires substantial mathematical reasoning, we use o4-mini. The temperature is set to 0.0 for all FMs in every setting, except for o4-mini, which is fixed at 1.0.
### D.2 Cost Estimate
Running the DGM-H for 100 iterations incurs a cost of approximately 33M tokens for the self-modification phase alone (excluding task evaluation). The total cost of an experiment therefore consists of the self-modification cost plus the cost of task evaluation. For the paper review and robotics reward design experiments, the evaluation cost per iteration is 0.506M tokens (0.5M tokens for paper review evaluation + 0.006M tokens for robotics reward design evaluation). Consequently, for a 100-iteration run ( Section 5 ), the estimated total cost is 33M tokens for self-modification plus 0.506M × 100 for evaluation, yielding a total of approximately 88.6M tokens.
A more granular break down of the task evaluation cost is:
### D.3 Improvement@k Metric
Let M M denote an initial meta agent, A A an initial task agent, and 𝒯 \mathcal{T} a fixed set of evaluation tasks. Let 𝒢 \mathcal{G} denote an agent-generation algorithm (e.g., DGM or DGM-H variants). The meta agent M M is held fixed and is allowed to generate up to k k new task agents by iteratively applying 𝒢 \mathcal{G} starting from A A .
Let 𝒜 ( k ) = { A 1 , A 2 , … , A k } \mathcal{A}^{(k)}=\{A^{1},A^{2},\dots,A^{k}\} denote the set of task agents generated by M M within k k modification steps. Each task agent A ′ ∈ { A } ∪ 𝒜 ( k ) A^{\prime}\in\{A\}\cup\mathcal{A}^{(k)} is evaluated on 𝒯 \mathcal{T} using a fixed evaluation procedure Evaluate ( ⋅ , 𝒯 ) \mathrm{Evaluate}(\cdot,\mathcal{T}) , where higher values indicate better performance.
We define the improvement@k metric as imp @ k ( M , A , 𝒢 , 𝒯 ) = max A ′ ∈ 𝒜 ( k ) ( M , A , 𝒢 ) Evaluate ( A ′ , 𝒯 ) − Evaluate ( A , 𝒯 ) , \mathrm{imp@}k(M,A,\mathcal{G},\mathcal{T})\;=\;\max_{A^{\prime}\in\mathcal{A}^{(k)}(M,A,\mathcal{G})}\mathrm{Evaluate}(A^{\prime},\mathcal{T})\;-\;\mathrm{Evaluate}(A,\mathcal{T}),
Intuitively, imp@k measures the maximum performance improvement that a fixed meta agent M M , operating under a specific agent-generation algorithm 𝒢 \mathcal{G} , can obtain by generating up to k k modified task agents starting from the initial task agent A A . Larger values of imp@k indicate stronger agent-generation capability under a fixed computational budget and generation procedure.
A limitation of imp@k is that it treats performance improvements as linear, without accounting for differences in difficulty across performance levels. In particular, improvements near saturation (e.g., increasing accuracy from 0.7 to 0.8) may be substantially harder to achieve than equivalent absolute gains at lower performance levels (e.g., from 0.0 to 0.1). As a result, imp@k may underestimate the significance of improvements achieved at higher performance regimes. However, this limitation does not affect the analyses presented in this work, as imp@k is used primarily for relative comparisons under matched initial conditions and fixed evaluation budgets, where all methods are subject to the same saturation effects ( Section 5.2 ).
### D.4 Transfer Agent Selection
To select agents for the transfer experiments ( Sections 5.2 and 5.3 ), we use a descendant growth criterion that favors agents which serve as strong stepping stones for subsequent improvements, rather than agents that are merely high-scoring themselves. Concretely, given the final archive at iteration t t , 𝒜 t = { a 0 , a 1 , … , a t } , \mathcal{A}^{t}=\{a_{0},a_{1},\dots,a_{t}\}, let α i \alpha_{i} denote the evaluation score of agent a i a_{i} on the source-domain validation set when available, and otherwise on the training set. Let parent ( j ) \mathrm{parent}(j) denote the parent of node j j in the archive tree, and let dist ( i , j ) \mathrm{dist}(i,j) be the number of edges on the unique path from i i to descendant j j .
We define the growth score of a candidate transfer node i i as the discounted average improvement achieved by its descendants relative to i i : G γ ( i ) = 1 | 𝒟 ( i ) | ∑ j ∈ 𝒟 ( i ) ( α j − α i ) γ dist ( i , j ) , G_{\gamma}(i)\;=\;\frac{1}{|\mathcal{D}(i)|}\sum_{j\in\mathcal{D}(i)}\bigl(\alpha_{j}-\alpha_{i}\bigr)\,\gamma^{\mathrm{dist}(i,j)}, where 𝒟 ( i ) \mathcal{D}(i) is the set of descendants of i i in the archive tree and γ ∈ ( 0 , 1 ] \gamma\in(0,1] controls how strongly we discount improvements that occur many generations after i i . Intuitively, G γ ( i ) G_{\gamma}(i) assigns higher weight to agents that reliably generate better descendants within fewer self-modification steps, which we treat as evidence of stronger agent-generation ability.
In our experiments, we set γ = 0.6 \gamma=0.6 and select the transfer agent with highest G 0.6 ( i ) G_{0.6}(i) . To reduce noise, we only consider nodes with at least 3 descendants.
## Appendix E Additional Results
This appendix presents additional qualitative and diagnostic results that complement the main findings. We first highlight the best task agents discovered by the DGM-H ( Section E.1 ). We then qualitatively analyze how the DGM-H improves task performance across different domains ( Section E.2 ) and how it develops meta-level capabilities that improves its ability to self-improve ( Section E.3 ). Next, we analyze the behavior of automatically discovered Olympiad-level math graders ( Section E.4 ). We also report preliminary experiments in which the DGM-H is allowed to modify its own parent selection mechanism, shedding light on the limits and potential of fully self-referential optimization ( Section E.5 ). All experiment logs are open-sourced in our codebase.
### E.1 Best Discovered Task Agents
We show portions of the diff patches that contribute to the task agent and are relevant to the domain. The full diff patches are open-sourced in our codebase.
#### E.1.1 Paper Review
Diff patches contributing to the best task agent discovered by the DGM-H ( Section 5.1 ) for paper review: ⬇
⬇
⬇
⬇
⬇
#### E.1.2 Robotics Reward Design
Diff patches contributing to the best task agent discovered by the DGM-H ( Section 5.1 ) for robotics reward design: ⬇
⬇
⬇
⬇
#### E.1.3 Olympiad-level Math Grading
Diff patches contributing to the best task agent, which we refer to as BetterGrader in Section E.4 , discovered by the DGM-H with transfer and from ProofAutoGrader ( Section 5.3 ) for Olympiad-level math grading: ⬇
⬇
⬇
⬇
⬇
⬇
### E.2 Qualitative: Improving Task Performance
Here we provide a qualitative view of how the DGM-H improves task performance over time by visualizing the archive trees and progress plots for one run in each setting ( Section 5 ). For each domain, we annotate key nodes in the archive with the code changes that affected the behavior of the task agent only for the domain being analyzed. Across diverse domains (i.e., paper review, robotics reward design, and Olympiad-level math grading), the DGM-H consistently demonstrates the ability to self-improve in meaningful ways ( Figures 7 , 8 and 9 ). Notably, many lineage paths leading to the final best-performing agent pass through intermediate nodes with lower performance, illustrating the benefits of open-ended search, which explores a diverse set of promising stepping stones rather than exclusively branching from the current best solution.