Language Agents as Optimizable Graphs — part 4/4
Cleaned-text code points [45029, 53258). Bibliography omitted here; source.md includes it.
Our TOT agent uses a tree-search strategy across a perfect binary tree with a depth of eight. Instead of constructing a graph of 2 9 − 1 2^{9}-1 nodes to represent this tree, the search is carried out through a chain of eight branching nodes. Each branching node is designed to generate two solutions from every input solution that it processes, effectively embodying the TOT strategy.
#### E.2.2 Hyper-parameters & Prompts
The candidate word generation prompt and the pruning prompt are adapted from the original TOT work ( Yao et al., 2023 ) and detailed in Table 7 . A clue is defined as a partial filling of the crossword, accompanied by its intended word description and specific position on the board.
Prompt purpose Prompt Candidate words generation prompt Let’s play a 5 x 5 mini crossword, where each word should have exactly 5 letters. {current board status} Unfilled: {Unfilled clues} Filled: {filled clues} Changed: {Changed clues} Suggestions: {suggestions generated by previous Reflection nodes} Given the current status, list all possible answers for unfilled or changed words, and your confidence levels (certain/high/medium/low), using the format ”h1. apple (medium)”. Use ”certain” cautiously and only when you are 100% sure this is the correct word. You can list more then one possible answer for each word. Pruning prompt Evaluate if there exists a five letter word of some meaning that fit some letter constraints (sure/maybe/impossible). Incorrect; to injure: w _ o _ g The letter constraint is: 5 letters, letter 1 is w, letter 3 is o, letter 5 is g. Some possible words that mean ”Incorrect; to injure”: wrong (w r o n g): 5 letters, letter 1 is w, letter 3 is o, letter 5 is g. fit! sure A person with an all-consuming enthusiasm, such as for computers or anime: _ _ _ _ u The letter constraint is: 5 letters, letter 5 is u. Some possible words that mean ”A person with an all-consuming enthusiasm, such as for computers or anime”: geek (g e e k): 4 letters, not 5 otaku (o t a k u): 5 letters, letter 5 is u sure Dewy; roscid: r _ _ _ l The letter constraint is: 5 letters, letter 1 is r, letter 5 is l. Some possible words that mean ”Dewy; roscid”: moist (m o i s t): 5 letters, letter 1 is m, not r humid (h u m i d): 5 letters, letter 1 is h, not r I cannot think of any words now. Only 2 letters are constrained, it is still likely maybe A woodland: _ l _ d e The letter constraint is: 5 letters, letter 2 is l, letter 4 is d, letter 5 is e. Some possible words that mean ”A woodland”: forest (f o r e s t): 6 letters, not 5 woods (w o o d s): 5 letters, letter 2 is o, not l grove (g r o v e): 5 letters, letter 2 is r, not l I cannot think of any words now. 3 letters are constrained, and _ l _ d e seems a common pattern maybe An inn: _ d _ w f The letter constraint is: 5 letters, letter 2 is d, letter 4 is w, letter 5 is f. Some possible words that mean ”An inn”: hotel (h o t e l): 5 letters, letter 2 is o, not d lodge (l o d g e): 5 letters, letter 2 is o, not d I cannot think of any words now. 3 letters are constrained, and it is extremely unlikely to have a word with pattern _ d _ w f to mean ”An inn” impossible Chance; a parasitic worm; a fish: w r a k _ The letter constraint is: 5 letters, letter 1 is w, letter 2 is r, letter 3 is a, letter 4 is k. Some possible words that mean ”Chance; a parasitic worm; a fish”: fluke (f l u k e): 5 letters, letter 1 is f, not w I cannot think of any words now. 4 letters are constrained, and it is extremely unlikely to have a word with pattern w r a k _ to mean ”Chance; a parasitic worm; a fish” impossible {clue} Suggestion Prompt You are playing a 5 x 5 mini crossword, where each word should have exactly 5 letters. Given the current status: {current board status } The target words are classified as Impossible Words, Correct Words, and Incorrect Words. --- Impossible Words: {impossible clues} Correct Words: {correct clues } Incorrect Words: {incorrect clues } Respond at most five sentences, one sentence per line. Do not include the phrase ”next time” in your response.
### E.3 HumanEval
#### E.3.1 The Node Optimization Method
For node optimization, we update each node after every four new problem executions. When addressing a new problem q q with graph G G , executing G ( q ) G(q) produces a program, denoted by s s , whose effectiveness is assessed against test examples associated with q q . The input-output pairs of the nodes generated during the evaluation of G ( q ) G(q) are classified as positive if s s passes the tests and as negative otherwise. We limit each node to include a maximum of four demonstration examples. Let n n be a node in the graph associated with a computational routine f n p f_{n}^{p} , which returns Python programs, parameterized by demonstration examples p p . During an optimization step of n n , that is, an application of I I as described in Section 2.4 , we assess whether to retain existing demonstration examples ( p n 1 p_{n}^{1} ) or to augment them with positive examples from the four most recent problems, subsequently randomly selecting up to four unique examples from this pool (denoted as p n 2 p_{n}^{2} ). More specifically, let Z Z be the set of the last ten inputs of node n n received when solving the first-seen problems. We select p n i p_{n}^{i} to update the demonstration examples of n n , where i = arg max i ∈ { 1 , 2 } ∑ z ∈ Z 𝟙 z ( f n p n i ( z , q z ) ) i=\argmax_{i\in\{1,2\}}\sum_{z\in Z}\mathbbm{1}_{z}(f_{n}^{p_{n}^{i}}(z,q_{z})) , 𝟙 z \mathbbm{1}_{z} determines whether a program passes the unit tests stated in z z , and q z q_{z} is the original graph input associated with z z .
The utility measure for Mini Crosswords experiments is defined as the best state word accuracy, as detailed in Section 3.2 . To reduce the variance in gradient estimation with the REINFORCE algorithm, we adjust the utility by subtracting a constant of 0.4 0.4 . For example, a perfectly completed solution results in a utility of 0.6 0.6 , while an empty solution yields a utility of − 0.4 -0.4 .
#### E.3.2 Hyper-parameters & Prompts
Table 8 shows the prompts used in our experiments following the principle of ReAct ( Yao et al., 2022 ) .
### E.4 GAIA
#### E.4.1 Agent Setting
We design different agents and swarms. Representative agents and swarms are visualized in Figure 9 .
#### E.4.2 Hyper-parameters & Prompts
We use GPT-4-Turbo for the experiments and design different node operations to solve the GAIA tasks. Table 9 and Table 10 show the prompts used in our experiments.
## Appendix F Resource Requirements
Table 11 presents the cost, token consumption, and time requirements of representative experiments.
## Appendix G Limitation and Future Work
In this paper, we focus on optimizing communication between agents, laying the groundwork for more extensive graph optimization in the future. For example, while current methods optimize edge connections between agents, the internal node topology of each agent is also crucial. Dynamically changing the topology may enhance task planning. Additionally, scaling up the agent framework is essential. When the number of agents exceeds 100, maintaining communication efficiency and system robustness becomes a significant challenge.
## Instructions for reporting errors
We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:
Click the "Report Issue" ( ) button, located in the page header.
Tip: You can select the relevant text first, to include it in your report.
Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.
Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .