Structured Scaling of AI Discovery Across Diverse Scientific Domains — part 4/12

Cleaned-text code points [47458, 62964). Bibliography omitted here; source.md includes it.

Even with robust evaluators, the present framework remains limited to problems for which feedback can be obtained automatically and inexpensively enough to support repeated queries. Many important scientific objectives fall outside this setting because their evaluation is subjective, experimentally costly, or available only through slow physical or wet-lab measurements. In addition, the evaluator-query budget is divided across C C , L L , and K K using fixed values, although our analyses show that the most effective allocation depends on both the task and the evolving state of the search. SimpleTES also relies primarily on continuous, fine-grained scores to guide refinement, making it less naturally suited to problems with sparse or binary feedback, such as formal theorem proving, where near-misses and unproductive attempts may receive indistinguishable scores. Extending evaluation-driven scaling to these settings will require adaptive allocation policies, richer semantic feedback extracted from failed attempts, partial proofs, and textual critiques, and interfaces to higher-fidelity simulators, laboratory robotics, or real-world measurements. Together, these advances could enable discovery loops that allocate effort dynamically and learn from heterogeneous forms of scientific feedback while preserving reliable verification.

### Code availability

The complete SimpleTES codebase, which includes the evaluation-driven search framework, task definitions, evaluators, benchmark assets, released best-result artifacts, and analysis scripts, is publicly available at https://github.com/wq-will/SimpleTES , providing a transparent and extensible resource for reproducing, inspecting, and building on the experiments reported here.

## Supplementary Information

### Contents

### A Authors

Contributors to this work are organized by working group.

Core contributors Haotian Ye † \dagger Haowei Lin Jingyi Tang Yizhen Luo Rahul Thapa Contributors Caiyin Yang Chang Su Rui Yang Ruihua Liu Rundao Li Zeyu Li Pengwei Sun Puheng Li Pan Lu Infrastructure Chong Gao Dachao Ding Guangrong He Miaolei Zhang Lina Sun Wenyang Wang Yuchen Zhong Zhuohao Shen Advising Bianxiao Cui Di He Jianzhu Ma Junfeng Li Hexi Baoyin Yejin Choi Stefano Ermon Xiaowen Chu Tongyang Li † \dagger Yuzhi Xu † \dagger James Zou † \dagger

† \dagger Correspondence : haotianye@stanford.edu , james@stanford.edu , xuyuzhi@wizardquant.com , and tongyangli@pku.edu.cn .

### B Related Work

#### B.1 Existing Evaluation-Driven Discovery Methods

Existing evaluation-driven discovery systems can be viewed as different instantiations of the TES policy π \pi introduced in Methods. Given an evaluated history of records ( y , r , m ) (y,r,m) , where V ⁡ ( y ) = ( r , m ) V(y)=(r,m) , the policy decides when to spend the next evaluator query, how to construct the next proposal x n x_{n} for the generator G G , and how the resulting feedback is stored for future proposals. This perspective separates three design choices that are often entangled in prior work: the allocation of the evaluation budget N N , the construction of proposals from historical feedback, and the location of adaptation, which may reside in an external archive, a prompt-level controller, the optimized artifact itself, or the generator parameters. It also clarifies how existing methods relate to the compact design space ( C , L , K , Φ ) (C,L,K,\Phi) studied in SimpleTES : most prior systems introduce sophisticated controllers or training mechanisms, whereas our focus is to isolate how scaling the evaluation-driven loop itself affects discovery.

##### AlphaEvolve [ 100 ] .

AlphaEvolve is an asynchronous evolutionary method with a frozen generator. Rather than following a single refinement chain, it spends the evaluation budget by repeatedly launching mutation jobs whenever sampling and evaluation capacity is available. Each proposal x n x_{n} is constructed from fixed problem context, one or more parent programs retrieved from an evolutionary database, previously evaluated solutions, rendered scores and execution feedback, and instructions asking the LLM to produce Search / Replace edits. Thus, the historical records ( y , r , m ) (y,r,m) are exposed to G G through an archive-driven prompting rule. After evaluation, the new record is inserted into the database, and future proposals resurface diverse high-performing programs using an island- or MAP-Elites-style archive policy. In the TES view, AlphaEvolve mainly adapts through the external archive and its resurfacing controller, while G G itself remains fixed. Its strong empirical performance comes with a complex policy π \pi whose precise archive-sampling heuristic is only partially specified in the public description.

##### OpenEvolve [ 6 ] .

OpenEvolve is an open-source implementation of AlphaEvolve-style evolutionary code discovery. It also treats TES as an asynchronous pipeline: generation, evaluation, and database updates proceed continuously rather than through synchronized refinement rounds. The proposal x n x_{n} is constructed from the problem description, selected evolved programs, evaluator scores, execution artifacts, and error feedback. OpenEvolve further supports LLM ensembles, multi-objective evaluation, MAP-Elites archives, island migration, checkpointing, and visualization. In our notation, its policy state is dominated by an external program database and the controller metadata used to sample parents, elites, diverse inspirations, and exploratory candidates. Compared with SimpleTES , OpenEvolve explores a much richer engineering space, but this also makes it harder to isolate which gains come from scaling evaluator queries N N and which come from archive heuristics, prompt engineering, or system-level design.

##### ShinkaEvolve [ 73 ] .

ShinkaEvolve makes the policy π \pi more explicitly adaptive. At each generation event, the controller first selects an island and then constructs a mutation proposal from that island’s archive. The proposal x n x_{n} may include a primary parent, inspiration programs sampled from top-performing and random archive entries, public performance metrics, textual evaluator feedback, and meta-scratchpad recommendations. The generator is also chosen from an LLM ensemble with sampled decoding settings, and the candidate may be requested as a diff edit, full rewrite, or crossover mutation. Before spending expensive evaluator queries, ShinkaEvolve can reject proposals using embedding-based novelty checks and an optional LLM novelty judge. After V V returns ( r , m ) (r,m) , the method updates per-island archives, offspring counts, model-selection statistics, and a meta-scratchpad summarizing recently successful strategies. Thus, adaptation occurs both through archive memory and through a controller that changes which model, parent, and mutation operator are used. In TES terms, ShinkaEvolve uses evaluation feedback not only to select better solutions, but also to continually reshape the proposal distribution induced by π \pi .

##### ThetaEvolve [ 155 ] .

ThetaEvolve uses batched decision events rather than purely asynchronous evolutionary updates. At each step, it samples many parents from a large program database and issues a batch of generator calls to a single LLM. Its proposal construction is comparatively lean: x n x_{n} typically contains task meta-information, code-replacement rules, and a sampled parent program, making the method close to iterative refinement over a large external database. Before expensive verification, ThetaEvolve performs early checks for malformed outputs, compile or runtime failures, invalid solutions, and duplicate programs. Valid children are evaluated by V V , inserted into the database, and used to reorganize future sampling. In its RL variant, the same batch also updates the generator parameters θ \theta using GRPO-style optimization and reward shaping. Therefore, unlike archive-only methods, ThetaEvolve can move part of the policy state into G G itself: successful mutation patterns are internalized by the model parameters rather than remaining only in the external history. This differs from our main setting, where G G is fixed in order to study the scaling behavior of evaluation-driven search more directly.

##### TTT-Discover [ 165 ] .

TTT-Discover explicitly combines evaluator-guided search with online model adaptation. Although the original paper describes the environment state as the current candidate solution, under the TES decomposition the reuse buffer, reuse statistics, and model parameters are all part of the effective policy state. At each rollout, the policy constructs a proposal by warm-starting from a previously discovered solution selected from a reuse buffer. The selection rule is PUCT-style: it uses rank-based priors, expansion counts, and the best reward achieved by descendants of a reused state. Prior actions can also be converted into natural-language context and inserted into the next proposal x n x_{n} . After evaluation, the new attempt is added to the buffer, reuse statistics are updated, and the model is trained online with an entropic RL objective that emphasizes high-reward discoveries. Thus, TTT-Discover is a clear example where TES feedback changes both the external memory used by π \pi and the generator parameters used by G G . Relative to SimpleTES , it places more emphasis on test-time training, whereas our main algorithm asks how far one can go by organizing evaluator queries with a fixed generator.

##### AdaEvolve [ 23 ] .

AdaEvolve introduces a hierarchical controller for deciding how to spend evaluator queries. Each iteration first chooses an island using a bandit rule, then decides whether that island should explore or exploit, and finally constructs a mutation proposal. In exploration mode, the policy samples parents more uniformly and pairs them with diverse inspirations, asking for more orthogonal changes. In exploitation mode, it samples stronger parents and asks for targeted refinements. When global stagnation is detected, a separate meta-guidance model analyzes the problem specification, evaluator, and recent failed attempts, then produces a high-level tactic that is injected into future proposals. After evaluation, AdaEvolve updates the island archive, improvement estimates, bandit rewards, visit counts, migration metadata, and the currently active tactic. In TES terms, AdaEvolve uses ( y , r , m ) (y,r,m) records not only to identify good candidates, but also to adapt the controller that allocates future budget across islands and search modes. This makes it a highly adaptive policy π \pi , but also one whose behavior depends on multiple interacting heuristics beyond evaluation scaling alone.

##### EvoX [ 82 ] .

EvoX operates on two timescales. The inner loop is a standard solution-discovery TES process: under an active strategy s t s_{t} , the policy constructs proposals by choosing parents, inspiration sets, and variation operators such as local refinement, free-form change, or structural divergence. The outer loop treats the strategy itself as an evolvable object. When progress over a sliding window falls below a stagnation threshold, a strategy-generator LLM receives the current population descriptor, prior strategies and their measured performance, a high-performing parent strategy, and inspirational strategies that worked in similar population states. It then proposes a new controller strategy. After each monitoring window, EvoX scores the deployed strategy, appends it to a strategy database, and may replace the active strategy without resetting the solution population. Under the TES framework, EvoX is notable because the optimized artifact is not only a solution y y , but also part of the policy π \pi that determines future proposal construction and archive updates. It therefore extends evaluation-driven scaling from solution search to controller search.

##### GEPA [ 2 ] .

GEPA is usually presented as prompt optimization for compound AI systems, but it also fits the TES template when the optimized artifact y y is a textual module, prompt, or agent scaffold. Each iteration selects a candidate from a Pareto frontier over per-example performance and then launches either a reflective mutation step or a merge step. To construct the next proposal, GEPA executes the selected candidate on a sampled minibatch, collects execution traces and evaluator-produced textual feedback, chooses which module to revise, and asks a reflection LLM to attribute successes and failures to specific prompt elements. A new candidate is admitted only if it improves on the minibatch; successful candidates are then evaluated more broadly and used to update the Pareto frontier. Thus, GEPA replaces parameter updates with textual reflection and Pareto-structured memory over prompt variants. In TES notation, its Φ \Phi is a reflection-based compressor from execution traces and feedback metadata m m into revised textual instructions. Compared with code-evolution systems, GEPA highlights that the same evaluation-driven loop applies beyond program synthesis, as long as candidate artifacts can be queried by an evaluator V V .

##### Summary.

Across these systems, evaluator feedback is the central information channel through which discovery improves. However, prior work often combines evaluation scaling with many other sources of improvement: ensemble generators, hand-designed prompt templates, archive heuristics, novelty filters, bandit controllers, meta-guidance, strategy evolution, or online model training. Our formulation abstracts these systems as policies π \pi that repeatedly construct proposals x n x_{n} from historical records ( y , r , m ) (y,r,m) and spend evaluator queries through V V . This abstraction motivates the simpler design studied in SimpleTES : fix the generator G G , make the evaluator budget N N explicit, and organize feedback-driven search through the compact dimensions ( C , L , K , Φ ) (C,L,K,\Phi) . Doing so does not subsume the full engineering richness of prior systems, but it provides a clean interface for studying what evaluation-driven scaling contributes by itself.

#### B.2 Self-evolving AI

The transition from static Large Language Models (LLMs) to self-evolving AI systems marks a paradigm shift toward Artificial Super Intelligence (ASI), where systems autonomously adapt their internal states, parameters, or architectural topologies based on interaction history and feedback. Following [ 40 ] , we summarize the methodology to develop self-evolving agents into 3 categories: reward-based self-evolution, imitation & demonstration learning, and evolutionary methods.

##### Reward-based Self-Evolution.

This line of research centers on closing the feedback loop through various reward signals to guide iterative improvement. Early frameworks like Reflexion [ 122 ] and AdaPlanner [ 135 ] leverage Textual Feedback , where the model generates natural language critiques to refine its future reasoning and memory. To reduce reliance on external supervision, Internal Reward mechanisms exploit the model’s own probability estimates or certainty to calibrate outputs [ 139 , 159 ] . Furthermore, External Rewards derived from sources outside the model, such as the environment [ 31 , 112 ] , majority voting [ 119 ] , or explicit rules [ 152 , 153 ] can also serve as an important signal for evolution.

##### Imitation & Demonstration Learning.