Automated Design of Agentic Systems — part 1/4

Cleaned-text code points [0, 15893). Bibliography omitted here; source.md includes it.

# Automated Design of Agentic Systems

###### Abstract

Researchers are investing substantial effort in developing powerful general-purpose agents, wherein Foundation Models are used as modules within agentic systems (e.g. Chain-of-Thought, Self-Reflection, Toolformer). However, the history of machine learning teaches us that hand-designed solutions are eventually replaced by learned solutions. We describe a newly forming research area, A utomated D esign of A gentic S ystems ( ADAS ), which aims to automatically create powerful agentic system designs, including inventing novel building blocks and/or combining them in new ways. We further demonstrate that there is an unexplored yet promising approach within ADAS where agents can be defined in code and new agents can be automatically discovered by a meta agent programming ever better ones in code. Given that most programming languages are Turing Complete, this approach theoretically enables the learning of any possible agentic system: including novel prompts, tool use, workflows, and combinations thereof. We present a simple yet effective algorithm named Meta Agent Search to demonstrate this idea, where a meta agent iteratively programs interesting new agents based on an ever-growing archive of previous discoveries. Through extensive experiments across multiple domains including coding, science, and math, we show that our algorithm can progressively invent agents with novel designs that greatly outperform state-of-the-art hand-designed agents. Importantly, we consistently observe the surprising result that agents invented by Meta Agent Search maintain superior performance even when transferred across domains and models, demonstrating their robustness and generality. Provided we develop it safely, our work illustrates the potential of an exciting new research direction toward automatically designing ever-more powerful agentic systems to benefit humanity. All code is open-sourced at https://github.com/ShengranHu/ADAS .

## 1 Introduction

Foundation Models (FMs) such as GPT ( OpenAI, 2024 ; OpenAI, 2022 ) and Claude ( Anthropic, 2024b ) are quickly being adopted as powerful general-purpose agents for agentic tasks that need flexible reasoning and planning ( Wang et al., 2024 ) . Despite recent advancements in FMs, solving problems reliably often requires an agent to be a compound agentic system with multiple components instead of a monolithic model query ( Zaharia et al., 2024 ; Rocktäschel, 2024 ) . Additionally, to enable agents to solve complex real-world tasks, they often need access to external tools such as search engines, code execution, and database queries. As a result, many effective building blocks of agentic systems have been proposed, such as chain-of-thought planning and reasoning ( Wei et al., 2022 ; Yao et al., 2023 ; Hu & Clune, 2024 ) , memory structures ( Zhang et al., 2024c ; Lewis et al., 2020 ) , tool use ( Schick et al., 2023 ; Qu et al., 2024 ) , and self-reflection ( Madaan et al., 2024 ; Shinn et al., 2023 ) . Although these agents have already seen significant success across various applications ( Wang et al., 2024 ) , developing these building blocks and combining them into complex agentic systems often requires domain-specific manual tuning and substantial effort from both researchers and engineers.

However, the history of machine learning reveals a recurring theme: manually created artifacts become replaced by learned, more efficient solutions ( Clune, 2019 ) over time as we get more compute and data ( Sutton, 2019 ) . An early example is from computer vision, where hand-designed features like HOG ( Dalal & Triggs, 2005 ) were eventually replaced by learned features from Convolutional Neural Networks (CNNs, Krizhevsky et al. (2012) ). More recently, AutoML methods ( Hutter et al., 2019 ) and AI-Generating Algorithms (AI-GAs, Clune (2019) ) have also demonstrated the superiority of learned AI systems compared to hand-designed AI systems. For example, the current best-performing CNN models come from Neural Architecture Search ( Elsken et al., 2019 ; Shen et al., 2023 ) instead of manual design; in LLM alignment, learned loss functions ( Lu et al., 2024a ) outperform most hand-designed ones such as DPO ( Rafailov et al., 2024 ) ; The AI Scientist ( Lu et al., 2024b ) demonstrates an automated research pipeline, including the development of novel ML algorithms; and an endless number of robotics learning environments can be automatically generated in works like OMNI-EPIC ( Faldor et al., 2024 ) , which demonstrate surprising creativity in generated environments and allow more efficient environment creation than the manual approach (see more examples in Section 5 ). Therefore, in this paper, we propose a new research question: Can we automate the design of agentic systems?

To explore the above research question, we describe a newly forming research area we call A utomated D esign of A gentic S ystems ( ADAS ), which aims to automatically invent novel building blocks and design powerful agentic systems ( Figure 2 ). We argue that ADAS may prove to be the fastest path to developing powerful agents, and show initial evidence that learned agents can greatly outperform hand-designed agents. Considering the tremendous number of building blocks yet to be discovered in agentic systems ( Section 5 ), it would take a long time for our research community to discover them all. Even if we successfully discover most of the useful building blocks, combining them into effective agentic systems for massive real-world applications would still be challenging and time-consuming, given the many different ways the building blocks can combine and interact with each other. In contrast, with ADAS, the building blocks and agents can be learned in an automated fashion. ADAS may not only potentially save human effort in developing powerful agents but also could be a faster path to more effective solutions than manual design.

Although a few existing works can be considered as ADAS methods, most of them focus only on designing prompts ( Yang et al., 2024 ; Fernando et al., 2024 ) , greatly limiting their ability to invent flexible design patterns in agents ( Section 5 ). In this paper, we show that there is an unexplored yet promising approach to ADAS where we can define the entire agentic system in code and new agents can be automatically discovered by a “meta” agent programming ever better ones in code. Given that most programming languages, such as Python, which we use in this paper, are Turing Complete ( Boyer & Moore, 1983 ; Ladha, 2024 ) , searching within a code space theoretically enables an ADAS algorithm to discover any possible agentic systems, including all components such as prompts, tool use, workflows, and more. Furthermore, with recent FMs being increasingly proficient in coding, we can use FMs as meta agents to create new agents in code for ADAS, enabling novel agents to be programmed in an automated manner.

Following the aforementioned ideas, we present Meta Agent Search in this paper as one of the first algorithms in ADAS that enables complete design in code space ( Figure 1 ). The core concept of Meta Agent Search is to instruct a meta agent to iteratively create interestingly new agents, evaluate them, add them to an archive that stores discovered agents, and use this archive to help the meta agent in subsequent iterations create yet more interestingly new agents. Similar to existing open-endedness algorithms that leverage human notions of interestingness ( Zhang et al., 2024a ; Lu et al., 2024c ) , we encourage the meta agent to explore interesting (e.g., novel or worthwhile) agents. To validate the proposed approach, we evaluate the proposed Meta Agent Search on: (1) the challenging ARC logic puzzle task ( Chollet, 2019 ) that aims to test the general intelligence of an AI system, (2) four popular benchmarks on reading comprehension, math, science questions, and multi-task problem solving, and (3) the transferability of discovered agents to held-out domains and models ( Section 4 ).

Our experiments show that the discovered agents substantially outperform state-of-the-art hand-designed baselines. For instance, our agents improve F1 scores on reading comprehension tasks in DROP ( Dua et al., 2019 ) by 13.6 /100 and accuracy rates on math tasks in MGSM ( Shi et al., 2023 ) by 14.4% . Additionally, they improve accuracy over baselines by 25.9% and 13.2% on GSM8K ( Cobbe et al., 2021 ) and GSM-Hard ( Gao et al., 2023 ) math tasks, respectively, after transferring across domains. The promising performance of our algorithm over hand-designed solutions illustrates the potential of ADAS in automating the design of agentic systems. Furthermore, the experiments demonstrate that the discovered agents not only perform well when transferring across similar domains but also exhibit strong performance when transferring across dissimilar domains, such as from mathematics to reading comprehension. This highlights the robustness and transferability of the agentic systems discovered by Meta Agent Search. In conclusion, our work opens up many exciting research directions and encourages further studies ( Section 6 ).

## 2 Automated Design of Agentic Systems (ADAS)

At the time of writing, the community has not reached a consensus on the definitions or terminologies of agents. Here, by agents we refer to agentic systems that involve Foundation Models (FMs) as modules in the workflow to solve tasks by planning, using tools, and carrying out multiple, iterative steps of processing ( Chase, 2024 ; Ng, 2024 ) . In this paper, we describe a newly forming research area Automated Design of Agentic Systems (ADAS). Similar to research areas in AI-GAs ( Clune, 2019 ) and AutoML ( Hutter et al., 2019 ) , such as Neural Architecture Search ( Elsken et al., 2019 ) , we formulate ADAS as an optimization process and identify three key components of ADAS algorithms ( Figure 2 ).

• Search Space : The search space defines which agentic systems can be represented and thus discovered in ADAS. For example, works like PromptBreeder ( Fernando et al., 2024 ) mutate only the text prompts of an agent, but their other components, such as workflow, remain the same. Thus, in these search spaces, agents that have a different workflow than the predefined one can not be represented. Existing works also explore search spaces such as graph structures ( Zhuge et al., 2024 ) and feed-forward networks ( Liu et al., 2023 ) .

• Search Algorithm : The search algorithm defines how ADAS algorithms explore the search space. Since the search space is often very large or even unbounded, the exploration-exploitation trade-off ( Sutton & Barto, 2018 ) should be considered. Ideally, the algorithm can both quickly discover high-performance agentic systems and avoid remaining stuck in a local optimum. Existing approaches include using Reinforcement Learning ( Zhuge et al., 2024 ) or an FM iteratively generating new solutions ( Fernando et al., 2024 ) as search algorithms.

• Evaluation Function : Depending on the application of the ADAS algorithm, we may consider different objectives to optimize, such as performance, cost, latency, or safety of agents. An evaluation function defines how to evaluate a candidate agent on those objectives. For example, to assess the agent’s performance on unseen future data, a simple method is to calculate the accuracy rate on the validation data for a task, which is commonly adopted in existing works ( Zhuge et al., 2024 ; Fernando et al., 2024 ) .

Although many search space designs are possible and some have already been explored ( Section 5 ), there is an unexplored yet promising approach where we can define the entire agentic system in code and new agents can be automatically discovered by a meta agent programming ever better ones in code. Searching within a code space theoretically enables the ADAS algorithm to discover any possible building blocks (e.g., prompts, tool use, workflow) and agentic systems that combine any of these building blocks in any way. This approach also offers better interpretability for agent design patterns since the program code is often readable, making debugging easier and enhancing AI safety. Additionally, compared to search spaces using networks ( Liu et al., 2023 ) or graphs ( Zhuge et al., 2024 ) , searching in a code space allows us to more easily build on existing human efforts. For example, it is possible to search within open-source agent frameworks like LangChain ( LangChainAI, 2022 ) and build upon all existing building blocks (e.g., RAG, search engine tools). Finally, since FMs are proficient in coding, utilizing a code search space allows us to leverage existing expertise from FMs during the search process. In contrast, search algorithms in custom search spaces, such as graphs, may be much less efficient due to the absence of these priors. Therefore, we argue that the approach of using programming languages as the search space should be studied more in ADAS.

## 3 Our Algorithm: Meta Agent Search

In this section, we present Meta Agent Search, a simple yet effective algorithm to demonstrate the approach of defining and searching for agents in code. The core idea of Meta Agent Search is to adopt FMs as meta agents to iteratively program interestingly new agents based on an ever-growing archive of previous discoveries. Although any possible building blocks and agentic systems can theoretically be programmed by the meta agent from scratch, it is inefficient in practice to avoid providing the meta agent any basic functions such as FM query APIs or existing tools. Therefore, in this paper, we define a simple framework (within 100 lines of code) for the meta agent, providing it with a basic set of essential functions like querying FMs or formatting prompts. As a result, the meta agent only needs to program a “forward” function to define a new agentic system, similar to the practice in FunSearch ( Romera-Paredes et al., 2024 ) . This function takes in the information of the task and outputs the agent’s response to the task. Details of the framework codes and examples of the agents defined with this framework can be found in Appendix C .

As shown in Figure 1 , the core idea of Meta Agent Search is to have a meta agent iteratively program new agents in code. The algorithm proceeds as follows: (1) The archive is (optionally) initialized with baseline agents such as Chain-of-Thought ( Wei et al., 2022 ) and Self-Refine ( Madaan et al., 2024 ; Shinn et al., 2023 ) . (2) Conditioned on the archive, the meta agent designs a new agent by generating a high-level description of the new idea for an agentic system and then implementing it in code. The design then undergoes two self-reflection ( Madaan et al., 2024 ; Shinn et al., 2023 ) steps by the meta agent to ensure it is novel. (3) The generated agent is evaluated using validation data from the target domain. If errors occur during evaluation, the meta agent performs a self-reflection step to refine the design, repeating this process up to five times if necessary. (4) Finally, the agent is added to the archive along with its evaluation metrics, and the process continues with the updated archive until the maximum number of iterations is reached. A pseudocode of the algorithm is provided in Appendix H .

Similar to existing open-endedness algorithms that leverage human notions of interestingness ( Zhang et al., 2024a ; Lu et al., 2024c ) , we encourage the meta agent to explore interestingly new (e.g., novel or worthwhile) agents based on an ever-growing archive of previous discoveries. Here, we calculate the performance (e.g., success rate or F1 score) as the metrics for the meta agent to maximize. The prompt and more details are presented in Appendix B .

## 4 Experiments