Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents — part 5/6
Cleaned-text code points [62835, 78230). Bibliography omitted here; source.md includes it.
• django__django-12419 • django__django-12663 • django__django-12741 • django__django-12858 • django__django-12965 • django__django-13012 • django__django-13023 • django__django-13028 • django__django-13033 • django__django-13089 • django__django-13109 • django__django-13112 • django__django-13121 • django__django-13128 • django__django-13158 • django__django-13195 • django__django-13212 • django__django-13297 • django__django-13315 • django__django-13343 • django__django-13344 • django__django-13363 • django__django-13401 • django__django-13406 • django__django-13410 • django__django-13417 • django__django-13449 • django__django-13512 • django__django-13513 • django__django-13516 • django__django-13551 • django__django-13568 • django__django-13569 • django__django-13590 • django__django-13658 • django__django-13670 • django__django-13741 • django__django-13786 • django__django-13794 • django__django-13807 • django__django-13809 • django__django-13810 • django__django-13820 • django__django-13821 • django__django-13837 • django__django-13925 • django__django-13933 • django__django-13964 • django__django-14007 • django__django-14011 • django__django-14017 • django__django-14034 • django__django-14053 • django__django-14089 • django__django-14122 • django__django-14140 • django__django-14155 • django__django-14170 • django__django-14238 • django__django-14311 • django__django-14315 • django__django-14349 • django__django-14351 • django__django-14373 • django__django-14376 • django__django-14404 • django__django-14434 • django__django-14493 • django__django-14500 • django__django-14534 • django__django-14539 • django__django-14559 • django__django-14580 • django__django-14608 • django__django-14631 • django__django-14672 • django__django-14725 • django__django-14752 • django__django-14765 • django__django-14771 • django__django-14787 • django__django-14792 • django__django-14855 ### E.3 Polyglot Tasks Initial 10 tasks for verifying basic functionality of a coding agent: • go__dominoes • cpp__all-your-base • python__dominoes • java__sgf-parsing • javascript__robot-name • rust__variable-length-quantity • python__beer-song • go__book-store • javascript__bottle-song • rust__bowling Additional 50 tasks for estimating general effectiveness of a coding agent: • javascript__queen-attack • rust__wordy • python__dot-dsl • java__satellite • cpp__diamond • rust__accumulate • go__error-handling • cpp__queen-attack • rust__poker • python__sgf-parsing • rust__react • java__ledger • go__connect • rust__macros • javascript__triangle • java__zipper • java__bowling • python__tree-building • javascript__say • java__wordy • python__food-chain • javascript__wordy • python__poker • javascript__grade-school • cpp__gigasecond • java__forth • python__dominoes • go__word-search • javascript__simple-linked-list • go__counter • java__react • javascript__ocr-numbers • python__scale-generator • java__go-counting • rust__doubly-linked-list • python__grade-school • javascript__forth • python__wordy • java__mazy-mice • cpp__bank-account • python__zipper • java__custom-set • java__rest-api • go__transpose • rust__gigasecond • rust__say • go__food-chain • rust__pig-latin • go__markdown • go__crypto-square ### E.4 SWE-bench State-of-The-Art At the time of writing this paper (16 April 2025), the highest performing, checked (i.e., the SWE-bench team received access to the system and were able to reproduce the patch generations), open-source entry on SWE-bench Verified is OpenHands + CodeAct v2.1 (claude-3-5-sonnet-20241022) ( Wang et al., 2024 ) , achieving 53.0%. Only considering the same subset of 200 tasks used by the DGM ( Section E.2 ), OpenHands + CodeAct v2.1 (claude-3-5-sonnet-20241022) achieves 51.0%. ### E.5 Polyglot Representative Agent Aider ( Gauthier, 2024 ) , a popular coding agent in the community, was published in Spring 2024. It has garnered over 33,000 stars on GitHub and has been continuously developed and tested against the Polyglot benchmark for over a year by human developers, primarily to evaluate its performance. Aider has also become a standard baseline for assessing the performance of different models, with the current top performers on the Polyglot benchmark being a mix of o3 (high) and GPT-4.1. We adopt a setup similar to that of the Polyglot leaderboard, with one key difference: the leaderboard reports pass@2 performance, where the agent can view feedback from ground-truth tests once. In contrast, we use a pass@1 setting, where the agent never sees the results of ground-truth tests, as we believe this more closely reflects realistic coding applications. ## Appendix F Best-Discovered Agents ### F.1 DGM on SWE-bench Diff patches contributing to the best agent discovered by the DGM on SWE-bench: ⬇ ### F.2 DGM on Polyglot Diff patches contributing to the best agent discovered by the DGM on Polyglot: ⬇ ## Appendix G Similar Target Functionality, Different Implementations For the same target functionality, the suggestions provided to the coding agents and the resulting implementations can differ significantly, leading to large variations in coding capability and benchmark performance. For example, consider nodes 6 and 24 in the DGM run on SWE-bench ( Figure 3 ). Both aimed to enhance the existing editor tool to support finer - grained file editing. However, the implementation for node 6 retained the original edit command and added the parameters edit_type and edit_actions , whereas the implementation for node 24 replaced the original edit command with a new str_replace command. Despite targeting similar functionality, the feature suggestions and thus the implementation details differed greatly, as did their performance. Node 6 achieved only 23.3%, while node 24 achieved 40.5%. Feature suggestion to obtain node 6: ⬇ Feature suggestion to obtain node 24: ⬇ Implementation of suggested feature to obtain node 6: ⬇ Implementation of suggested feature to obtain node 24: ⬇ ## Appendix H Case Study: Solving Hallucination The DGM can be used to optimize objectives beyond just coding, as discussed as a potential direction for future work in Section 5 . In this section, we show that the DGM can address hallucinations of tool use by FMs. Through feedback from evaluation logs, the DGM improves hallucination detection mechanisms and ultimately discovers ways to resolve these hallucinations. We provide a more detailed discussion of when these hallucinations occur, the DGM setup, and the resulting solutions. Finally, we present an example of objective hacking, where the agent optimizes for the quantifiable metric rather than truly fulfilling the spirit of the task and solving the underlying problem. When hallucination occurs. When the input context to Claude 3.5 Sonnet (New) (Claude) is very long, including all previous tool use inputs and outputs, we found that Claude tends to hallucinate tool usage. Instead of actually making a tool call, it outputs the tool input and output as plain text. This is an example of Claude hallucinating tool use, claiming that the Bash tool was used to run tests and that the tool output suggests that all tests passed, but Claude did not invoke any tools, and the tool results were entirely hallucinated in plain text: ⬇ DGM setup. We run this experiment on SWE-bench. Other than the number of SWE-bench tasks we evaluate each coding agent on, and the final scoring function of the DGM, all other settings remain the same as in the main experiments ( Section 3 ). The initial coding agent and all agents generated during the DGM run are evaluated on only 3 SWE-bench tasks. We use just 3 tasks because if hallucination occurs in one, it is likely to occur in others as well. Hence, 3 tasks are sufficient to provide the DGM with a reliable signal regarding whether tool use hallucination is still present. Solve hallucination score. We score each generated agent based on how effectively it prevents hallucinations, with higher scores indicating fewer hallucinations and better performance. For each SWE-bench task the agent is evaluated on, the score has two components: 0 - 1 based on whether hallucination occurred in the logs, and 1 - 2 (only added and accounted for after no hallucination occurs in the logs) if tools were used in all Claude calls. The final solve hallucination score is computed as the average across all evaluated tasks. To detect hallucination in the logs, we insert special tokens when actual tool use occurs and then check if these tokens appear in the model’s plain-text output, indicating that the model mimicked tool use without actually invoking any tools. Importantly, the hallucination checking functions are hidden from the coding agent during self-modification. We found that objective hacking (i.e., optimizing for the measurable objective instead of truly solving the underlying problem) occurs more frequently when these functions are not hidden. Results. We run the DGM for 150 iterations. In this run, the DGM discovers two promising solutions: node 96, which achieves the second highest score of 1.67, and node 114, which achieves a perfect score of 2.0 ( Figure 8 ). Node 96 addresses hallucination to some extent, achieving a score of 1.67 without objective hacking. In the lineage leading to node 96, the DGM first improves the tool transaction logging mechanisms to better detect when hallucination occurs. It then explores various strategies to solve it, such as removing hallucinated tool use from the output text when it determines that the tool was not actually invoked. Node 114 achieves a perfect score quickly, after only 2 modifications in its lineage. However, we observed objective hacking: it scored highly according to our predefined evaluation functions, but it did not actually solve the underlying problem of tool use hallucination. In the modification leading up to node 114 (see below), the agent removed the logging of special tokens that indicate tool usage (despite instructions not to change the special tokens), effectively bypassing our hallucination detection function. ⬇ In conclusion, the DGM can be applied beyond the coding domain, and we highlighted a case of objective hacking. Similar to reward hacking in reinforcement learning ( Skalse et al., 2022 ) , objective hacking occurs when a system optimizes for a predefined, quantifiable objective rather than fulfilling the spirit of the task or solving the intended problem. This observation supports arguments made in prior works ( Zhang et al., 2024b ; Faldor et al., 2025 ) , which suggest that optimizing quantitative measures often leads to undesirable or pathological outcomes, and aligns with Goodhart’s law ( Strathern, 1997 ) – "When a measure becomes a target, it ceases to be a good measure." ## Appendix I Additional Safety Discussion Any advancement that increases the autonomous capabilities of AI systems introduces its own set of safety considerations ( Bengio et al., 2024 ) , especially for systems that improve in an open-ended way ( Ecoffet et al., 2020 ; Clune, 2019 ) . Section 5 discusses these concerns and outlines concrete, actionable steps for mitigating them. We call for much more research into and discussion regarding AI safety, including deep thought and discussion amongst all stakeholders in society on the complicated question of what exactly counts as safe AI. We are confident the work we have done was never unsafe ( Section 5 ), but scaled up versions of it could be. As with all transformative technologies, the ultimate impact of such AI systems remains deeply uncertain, and good arguments can be made both for the case that it will bring about tremendous good and tremendous harm. These uncertainties highlight the need for sustained, inclusive, and multidisciplinary discussion (not only from current experts but also from a wider and more diverse community) on how to navigate these developments. ## Appendix J Additional Future Work Directions While this paper has shown the potential of the Darwin Gödel Machine in iteratively improving coding agents via open-ended exploration and empirical validation, several extensions could address current limitations and push AI beyond its already growing role in inspiring culture and advancing science. The following directions outline promising avenues for further research. Autonomously Improving the Open-ended Exploration Process. In this version of the DGM, the open-ended exploration process described in Section 3 is kept fixed, which might hence impede the system’s self-acceleration potential. This design choice was made due to limited computational budget. If we were to evolve this part of the algorithm, it could require exponentially more compute to identify processes that yield the same improvements shown in Section 4.4 . Nevertheless, since the open-ended exploration loop itself is implemented in code, it can in principle be edited and improved by a coding agent. There are many possible implementations of open-ended exploration, for example, using alternative search mechanisms that balance exploration and exploitation ( Herr et al., 2025 ) , keeping only the most interesting agents in the archive ( Faldor et al., 2025 ) , or leveraging the generated agent population as an ensemble ( Samvelyan et al., 2024 ) . A promising future work direction is to allow the agent to modify the open-ended exploration process, thereby autonomously improving not only its own capabilities but also the meta-process that allocates limited compute to drive self-improvement and self-acceleration. Role of Humans in Autonomous AI Systems. In the current formulation of the DGM, proposed self-modifications are autonomously evaluated without any human intervention. However, as autonomous systems grow in complexity and influence, the question of how humans should remain involved becomes increasingly pressing. Should human oversight be framed as an optimization objective, incorporated through techniques such as reinforcement learning from human feedback ( Ouyang et al., 2022 ) , or distilled into FMs that act as preference judges ( Bai et al., 2022 ) ? Each of these approaches raises challenges in terms of scalability, reliability, and alignment with evolving human values. The role of humans in guiding, constraining, or co-evolving with autonomous AI remains an open question. Exploring this dynamic is a promising avenue for future research, as it touches not only on technical feasibility but also on broader philosophical and societal considerations. DGM with Advanced Foundation Models. Recent FMs have advanced dramatically, enabling scaffolds to become simpler on current coding benchmarks ( Yang et al., 2024 ) . It is possible that in some settings, like current coding benchmarks, certain engineering efforts in scaffolding might be downplayed by the improvement of the FMs. However, many scaffolding components (advanced tools, parallel workflows, external memory, proxy verification, etc.) still fundamentally can not be internalized by FMs and will be essential for more complex real-world tasks beyond today’s benchmarks. Future work to explore how different components in agents will emerge with different FMs could be a promising direction.