Automated Design of Agentic Systems — part 3/4

Cleaned-text code points [30870, 46533). Bibliography omitted here; source.md includes it.

Existing Attempts to ADAS. There are two categories of works that attempt ADAS: those focused on learning better prompts and those that learn more components beyond prompts. Most works fall into the first category, where FMs are used to automate prompt engineering, primarily enhancing the phrasing of instructions to improve reasoning ( Yang et al., 2024 ; Fernando et al., 2024 ; Zhou et al., 2024a ; Yuksekgonul et al., 2024 ) . However, these prompts are often domain-specific and difficult to generalize. Some works optimize role definitions within prompts ( Yuan et al., 2024 ; Chen et al., 2023c ; Chen et al., 2023b ; Wu et al., 2023 ) , as assigning personas or roles to agents has been shown to be beneficial ( Xu et al., 2023 ) . Although tuning prompts can improve performance, other components remain fixed, limiting the space of agents that can be discovered. The second category, which is less explored, involves learning additional components such as workflows, often representing agents as networks or graphs. In these formulations, the FM with a certain prompt is considered a transformation function for text on nodes, and the information flow of the text is considered as edges. For example, DyLAN ( Liu et al., 2023 ) uses FMs to optimize connections between nodes in a network, DSPy ( Khattab et al., 2024 ) and Trace ( Cheng et al., 2024 ) optimizes across the Cartesian product of a set of possible nodes, and GPT-Swarm ( Zhuge et al., 2024 ) uses reinforcement learning to optimize node connections. Although these approaches optimize workflows, many components like tool usage remain fixed. AgentOptimizer ( Zhang et al., 2024b ) learns the tools used in agents, AutoFlow ( Li et al., 2024 ) proposes a new language to optimize workflow, Agent Symbolic Learning ( Zhou et al., 2024b ) attempts to learn prompts, tools, and workflows together. While these works share similar motivations to learn more components in agents, they either fail to cover all possible designs in agentic systems or have harder search spaces for search algorithms. In contrast, our work represents all components in code, allowing all possible designs in agentic systems and resulting in a promising search space for FM-guided search, as coding tasks are one of the most important tasks in FMs’ training.

## 6 Discussion and Conclusion

Safety Considerations. While it is highly unlikely that model-generated code will perform overtly malicious actions in our current settings with the Foundation Models (FMs) we employ, such code could still act destructively due to limitations in model capability or alignment ( Rokon et al., 2020 ; Chen et al., 2021 ) . To address these risks, we have implemented safety measures including containerized execution of all generated code in secure, isolated environments, thorough manual inspections to verify the absence of harmful behaviors, and clear warnings in our codebase to alert users to potential risks. These practices align with established safety standards in the literature, such as those in SWE-Bench ( Jimenez et al., 2024 ) and Voyager ( Wang et al., 2023a ) , which similarly prioritize controlled execution environments.

The proposed Automated Design of Agentic Systems (ADAS) introduces a novel area in AI-GA research, potentially accelerating the development of Artificial General Intelligence (AGI) beyond current manual approaches ( Clune, 2019 ) . This raises broader questions about advancing AI capabilities, a topic extensively debated in prior works ( Clune, 2019 ; Ecoffet et al., 2020 ; Bostrom, 2002 ; Yudkowsky et al., 2008 ; Bengio et al., 2024 ) , though beyond this paper’s scope. We argue that publishing this work is net beneficial. It reveals that powerful ADAS algorithms can be easily programmed using API access to FMs, without requiring expensive hardware like GPUs, informing the community of their accessibility and implications. Moreover, ADAS can enhance safety in agentic systems by automating the design of explicit, interpretable workflows, reducing the risk of malicious behavior through greater controllability and auditability.

We believe the discussion on ADAS and its safety impact is timely given the growing adoption of agentic systems in real-world applications ( Turow, 2024 ) , where ADAS can streamline the creation of safe, reliable agents, amplifying AI’s potential to benefit humanity in domains like health and economics ( Amodei, 2024 ) . Furthermore, as self-improving AI systems become prominent ( Clune, 2019 ; Fernando et al., 2024 ; Lu et al., 2024a ; Zelikman et al., 2022 ) , their continued development appears inevitable. By sharing this work, we aim to inspire further research into safe-ADAS algorithms—potentially incorporating mechanisms like Constitutional AI ( Bai et al., 2022 ) —to ensure that advancements in AI-GA and self-improving AI yield systems that are both powerful and aligned with human values, ultimately fostering safer AI development.

Future Work. Our work also opens up many future research directions: • Higher-order ADAS. Since the meta agent used in ADAS to program new agents in code is also an agent, ADAS can become self-referential where the meta agent can be improved through ADAS as well. It would be an exciting direction to have a higher order of meta-learning to allow the learning of the meta agent and even the meta-meta agent, etc. ( Lu et al., 2023 ; Schmidhuber, 1987 ; Schmidhuber, 2003 ; Zelikman et al., 2024 )

• Online Continual Learning. As agents are deployed, they will receive vast amounts of feedback from both task environments and users. Continuously improving agents based on this extensive feedback is challenging for human developers. However, with ADAS automating the design and enhancement of agents, online continual learning becomes feasible post-deployment.

• Multi-objective ADAS. We only consider one objective (i.e., performance) to optimize in this paper, but in practice, multiple objectives are often considered, such as cost, latency, and robustness of agentic systems ( Hu et al., 2021 ; Huang et al., 2023 ) . Thus, integrating multi-objective search algorithms ( Deb et al., 2002 ) in ADAS could be promising.

• Towards a Better Understanding of FMs. Works from Neural Architecture Search ( Huang et al., 2023 ) show that by observing the emerged architecture, we could gain more insights into Neural Networks. In this paper, we also gained insights about FMs from the results. For example, the best agent with GPT-3.5 involves a complex feedback mechanism, but when we transfer to other advanced models, the agent with a simpler feedback mechanism but more refinement becomes a better agent ( Section 4.3 ). This shows that GPT-3.5 may have a worse capability in evaluating and refining the answers, so it needs a complex feedback mechanism for better refinement, while other advanced models benefit more from a simpler feedback mechanism.

• More complex domains. Currently, we only evaluate Meta Agent Search on single-step QA tasks in this paper. It would be interesting to extend the method to more complex domains, such as real-world applications involving multi-step interaction with complex environments.

• Seeding ADAS with more existing building blocks. Although we can theoretically allow any components in agentic systems to be programmed from scratch in the code space, it is not efficient in practice. Therefore, it would be interesting to explore ADAS by standing on the shoulders of existing human efforts, such as search engine tools, RAG ( Lewis et al., 2020 ) , or functions from existing agent frameworks like LangChain ( LangChainAI, 2022 ) . Additionally, it is interesting to support multi-modal capabilities (e.g. vision) in FMs or allow different FMs to be available in agentic systems. This will enable the meta agent to choose from different FMs flexibly according to the difficulty of the instruction and whether data privacy is a priority.

• Novelty search algorithms. In Meta Agent Search, the design of the search algorithm is relatively simple, focusing solely on exploring interesting new designs. A more careful design of the search algorithm can be a promising future direction. For example, one could incorporate more sophisticated ideas from Quality-Diversity ( Mouret & Clune, 2015 ; Cully & Demiris, 2017 ) , AI-generating ( Clune, 2019 ) , and Open-ended Algorithms ( Faldor et al., 2024 ; Zhang et al., 2024a ; Stanley & Lehman, 2015 ; Stanley et al., 2019 ) . One could also include more classic approaches to balance exploration and exploitation ( Sutton & Barto, 2018 ; Liu et al., 2024 ) .

• More Intelligent Evaluation Functions. In this work, we simply evaluate discovered agents on the evaluation set and use the numerical performance results. However, this approach is both expensive and misses a lot of information. A promising future direction is to enable the meta agent to analyze detailed running logs during the evaluation, which contain rich information on the failure and success modes for better debugging and improving agentic systems ( Zhou et al., 2024b ) . Also, many tasks involve subjective answer evaluations ( Chiang et al., 2024 ; Lu et al., 2024b ) that do not have ground-truth answers. It is also important to design novel evaluation functions in ADAS to address these tasks. Finally, in this work, we targeted only one domain during the search. It would be interesting to explore whether ADAS algorithms can design even better generalist agents when specifically searching for agents capable of performing well across multiple domains.

• Understanding the emergence of complexity from human organizations. Beyond potentially saving researchers’ efforts and improving upon the manual design of agentic systems, the research in ADAS is also scientifically intriguing as it sheds light on the origins of complexity emerging from human organization and society. The agentic system is a machine learning system that operates primarily over natural language—a representation that is interpretable to humans and used by humans in constructing our organization and society. Thus, there is a close connection between agentic systems and human organizations, as shown in works incorporating the organizational structure for human companies in agents ( Hong et al., 2023 ) or simulating a human town with agents ( Park et al., 2023 ) . Therefore, the study in ADAS may enable us to observe how to create a simple set of conditions and have an algorithm to bootstrap itself from simplicity to produce complexity in a system akin to human society.

Conclusion. In this paper, we propose a new research problem, Automated Design of Agentic Systems (ADAS), which aims to automatically invent novel building blocks and design powerful agentic systems . We demonstrated that a promising approach to ADAS is to define agents in code, allowing new agents to be automatically discovered by a “meta” agent programming them in code. Following this idea, we propose Meta Agent Search, where the meta agent iteratively builds on previous discoveries to program interesting new agents. The experiments show that Meta Agent Search consistently outperforms state-of-the-art hand-designed agents across an extensive number of domains, and the discovered agents transfer well across models and domains. Overall, our work illustrates the potential of an exciting new research direction toward full automation in developing powerful agentic systems from the bottom up.

#### Acknowledgments

This work was supported by the Vector Institute, the Canada CIFAR AI Chairs program, grants from Schmidt Futures and Open Philanthropy, an NSERC Discovery Grant, and a generous donation from Rafael Cosman. We thank Jenny Zhang, Rach Pradhan, Ruiyu Gou, Nicholas Ioannidis, and Eunjeong Hwang for insightful discussions and feedback.

## Supplementary Material

## Table of Contents

## Appendix A Generalization and Transferability

In this section, we present more details of the experiments in Section 4.3 and the complete results of transferring agents across different domains.

For the results shown in Table 3 , we use “gpt-4o-2024-05-13” for GPT-4, “claude-3-haiku-20240307” for Claude-Haiku, and “claude-3-5-sonnet-20240620” for Claude-Sonnet.

We transfer the discovered agent from the MGSM (Math) domain to other math domains to test whether the invented agents can generalize across different domains. Similarly, we test the top 3 agents from MGSM and transfer them to (1) four popular math domains: GSM8K ( Cobbe et al., 2021 ) , GSM-Hard ( Gao et al., 2023 ) , SVAMP ( Patel et al., 2021 ) , and ASDiv ( Miao et al., 2020 ) and (2) three domains beyond math adopted in Section 4.2 . As shown in Table 4 , we observe a similar superiority in the performance of Meta Agent Search compared to baselines. More surprisingly, we observe that agents discovered in the math domain can be transferred to non-math domains ( Table 5 ). While the performance of agents originally searched in the math domain does not fully match that of agents specifically designed for the target domains, they still outperform (in Reading Comprehension and Multi-task) or match (in Science) the state-of-the-art hand-designed agent baselines. These results illustrate that Meta Agent Search can discover generalizable design patterns and agentic systems.

## Appendix B Prompts

We use the following prompts for the meta agent in Meta Agent Search. Variables in the prompts that vary depending on domains and iterations are highlighted .

We use the following system prompt for every query in the meta agent.

We use the following prompt for the meta agent to design the new agent based on the archive of previously discovered agents.

The domain descriptions are available in Appendices E and 4 and the framework code is available in Appendix C . We use the following prompt to instruct and format the output of the meta agent. Here, we collect and present some common mistakes that the meta agent may make in the prompt. We found it effective in improving the quality of the generated code. These formatting prompts are inspired by Lu et al. (2024a) .

After the first response from the meta agent, we perform two rounds of self-reflection to make the generated agent novel and error-free ( Shinn et al., 2023 ; Madaan et al., 2024 ) .

When an error is encountered during the execution of the generated code, we conduct a reflection and re-run the code. This process is repeated up to five times if errors persist. Here is the prompt we use to self-reflect any runtime error:

## Appendix C Framework Code

In this paper, we provide the meta agent with a simple framework to implement basic functions, such as querying Foundation Models (FMs) and formatting prompts. The framework consists of fewer than 100 lines of code (excluding comments). In this framework, we encapsulate every piece of information into a namedtuple Info object, making it easy to combine different types of information (e.g., FM responses, results from tool function calls, task descriptions) and facilitate communication between different modules. Additionally, in the FM module, we automatically construct the prompt by concatenating all input Info objects into a structured format, with each Info titled by its metadata (e.g., name, author). Throughout the appendix, we renamed some variables in the code to match the terminologies used in the main text.

With the provided framework, an agent can be easily defined with a “forward” function. Here we show an example of implementing self-reflection using the framework.

## Appendix D Experiment Details for ARC Challenge