Structured Scaling of AI Discovery Across Diverse Scientific Domains — part 5/12

Cleaned-text code points [62964, 77722). Bibliography omitted here; source.md includes it.

Stabilizing evolution by mimicking high-quality exemplars, imitation learning provides a prescriptive path to capability enhancement. This paradigm has evolved from human-centric demonstrations to Self-Generated Demonstrations , where agents like STaR [ 167 ] and Quiet-STaR [ 166 ] bootstrap their reasoning by fine-tuning on self-produced successful trajectories. Recent advancements also explore Cross-Agent Demonstration , enabling knowledge transfer within multi-agent systems where agents learn from the collective "experience library" of more capable peers [ 172 , 138 ] .

##### Population-based & Evolutionary Methods.

This paradigm focuses on agent improvement through evolutionary operators or iterative self-confrontation. Learning from evolution applies genetic operators—selection, mutation, and crossover—to discover improved capabilities in code, architecture, or parameters, exemplified by AlphaEvolve [ 100 ] . Alternatively, Self-Play creates a dynamic learning process where agents improve by interacting with versions of themselves, such as Absolute Zero [ 171 ] and R-Zero [ 57 ] .

In summary, these evolutionary paradigms collectively enhance the reasoning capabilities of LLMs by strategically allocating additional computation and adapting to feedback during either training or inference. Building upon these foundations, SimpleTES extends the Learning from Evolution paradigm by leveraging explicit evaluator feedback to autonomously break through the performance upper bounds of difficult, open-ended scientific discovery problems without requiring any expert demonstrations. Furthermore, our training process aligns with the Self-Generated Demonstration approach by treating high-quality, long-horizon TES trajectories as a natural form of supervision, enabling the model to internalize global exploration strategies from its own trial-and-error experiences.

#### B.3 LLM for Scientific Discovery

The application of LLMs in scientific discovery is undergoing a fundamental paradigm shift, transitioning from passive assistance tools to autonomous research agents capable of end-to-end scientific investigation. Modern AI systems have moved towards closed-loop frameworks that independently propose hypotheses, design and execute experiments, and perform automated reviews. This rapid progress is driven by the scaling of search compute, the development of end-to-end autonomous systems, and the establishment of rigorous scientific benchmarks.

##### LLM based scientific discovery systems.

Several pioneering systems have demonstrated the potential for automating various stages of AI research. The AI Scientist [ 85 ] provides a pipeline for the research lifecycle, including phases for ideation, manuscript generation, and automated peer review. Other platforms, such as the Fully Automated Research System ( FARS ) [ 4 ] , utilize multi-agent topologies to generate numerous research artifacts in a fully automated manner. Additionally, the AutoResearch framework [ 68 ] implements an execution loop where agents autonomously modify code and validate performance improvements in real-time.

##### Scientific benchmarks for LLMs.

As LLM capabilities extend into complex engineering and reasoning, a new generation of benchmarks has emerged to evaluate their scientific proficiency. MLE-Bench [ 25 ] and PaperBench [ 134 ] assess the system’s capacity for high-level machine learning engineering and research reproducibility. In terms of performance optimization, KernelBench [ 103 ] targets systems-level GPU kernel design, while AlgoTune [ 108 ] focuses on accelerating algorithm execution in wall-clock time. Additionally, specialized tasks have been introduced to evaluate domain-specific expertise, such as symbolic regression [ 124 ] , quantum circuit design [ 160 ] , medical diagnostic reasoning [ 177 ] , and comprehensive physical research capabilities [ 94 ] .

##### Important results proposed by LLM systems.

Concurrent with the development of research systems and benchmarks, several significant breakthroughs have been achieved in fundamental sciences through LLM-driven methodologies. Using the AlphaEvolve [ 100 ] framework, researchers have conducted large-scale mathematical exploration, providing insights into complex problems such as functional inequalities and conjectures [ 42 ] . Google DeepMind has reported progress with AlphaProof [ 58 ] , which reached competitive standards in mathematical olympiads by combining LLMs with formal verification. Other developments include AlphaGeometry [ 145 ] for automated geometry solving and FunSearch [ 113 ] , which has been used to discover new solutions for combinatorial problems and surpass certain human-designed heuristics in algorithm optimization.

In summary, these systems and benchmarks illustrate a trajectory where AI is evolving from a research assistant into a self-directed collaborator. Building upon this foundation, our proposed SimpleTES framework advances this evolution by formalizing Test-Time Evaluation-driven Scaling (TES) to achieve generalizable breakthroughs in open-ended scientific problems without expert demonstrations.

#### B.4 Test-Time Scaling

Researchers have increasingly shifted their focus toward eliciting stronger intelligence during inference due to the gradually diminishing gains from scaling pretraining data and parameters. Inspired by cognitive science theories wherein complex problems trigger deeper, deliberate "System 2" thinking, Test-Time Scaling (TTS)—also referred to as test-time computing—allocates additional computation during inference to boost task performance and problem-solving capabilities. Following the unified hierarchical framework [ 170 ] , we can categorize TTS methods based on "what to scale" into four distinct paradigms: Parallel Scaling, Sequential Scaling, Hybrid Scaling, and Internal Scaling.

##### Parallel Scaling.

While LLMs traditionally generate a single response per query, parallel scaling enhances test-time performance by generating multiple candidate outputs concurrently and aggregating them into a final answer. A prominent line of research within this paradigm centers on the concept of Self-Consistency, which leverages sampling a diverse set of reasoning paths and applying majority voting to reduce generation variance and mitigate hallucinations [ 154 , 19 , 130 ] . Other approaches leverage multi-agent frameworks [ 64 ] or structured generation strategies [ 150 ] to further broaden the coverage and diversity of potential solutions.

##### Sequential Scaling.

In contrast to parallel generation, sequential scaling explicitly directs later computations based on intermediate steps, updating partial solution states iteratively. Since many complex tasks require deep deliberation rather than immediate pattern matching, this paradigm mimics a step-by-step refinement process. Notable implementations include Self-Refine [ 90 ] , which allows the model to iteratively critique and improve its own initial drafts without external training data. Other prominent works include ReAct [ 162 ] , which scales the capability of LLMs by sequentially interleaving reasoning and acting.

##### Hybrid Scaling.

Hybrid scaling exploits the complementary benefits of both parallel and sequential approaches. By generating multiple hypotheses in parallel (divergent thinking) and sequentially filtering or refining them (convergent thinking), hybrid methods can deeply explore promising reasoning paths while mitigating the risk of missing the correct answer. The Tree of Thoughts (ToT) framework [ 161 ] is a quintessential example, allowing the model to branch out at decision points and prune unpromising paths. This concept has been extensively expanded upon by Graph of Thoughts [ 12 ] and various implementations of Monte Carlo Tree Search (MCTS) [ 36 , 141 ] .

##### Internal Scaling.

Internal scaling represents a recent paradigm shift wherein the model autonomously determines how much test-time computation to allocate without relying on external, human-guided prompting architectures. Through specific training procedures, these models learn internal policies that dictate when to continue reasoning and when to halt. Prime examples of this autonomous scaling include OpenAI’s o1 and o3 models [ 63 ] and DeepSeek-R1 [ 51 ] .

In summary, these four paradigms collectively enhance the reasoning capabilities of LLMs by strategically allocating additional computation during inference. Building upon this foundation, SimpleTES further extends the conventional TTS paradigm to Test-Time Evaluation-driven Scaling (TES), relying on explicit evaluator feedback to enable the resolution of more difficult, open-ended scientific discovery problems.

### C Theoretical Modeling of SimpleTES

This section gives theoretical insights on the design choice of Definition 0.1 . Specifically, we will start by modeling the fundamental limitation of sequential refinement policy ( 1 , L , 1 , Φ ) (1,L,1,\Phi) , and then justify the necessity of global width C C by a theorem, and then explain the importance of local sample size K K . Since the goal is to understand the scaling effect of these dimensions, throughout the section we use a simplified mathematical model based on the Pólya Urn model [ 91 ] and do not consider the complexity induced by Φ \Phi . Readers might treat this section as a theoretical rewrite of SimpleTES in Methods.

A standard pure sequential refinement iteratively improves a solution conditioned on experiences from all previous candidates and evaluation results. Arguably, even equipped with an idealized generator with perfect attention and an infinite context window, this policy remains sub-optimal due to a fatal mismatch with the nature of open-ended problems. On one hand, solving a complex open-ended problem requires multidimensional coverage: a high-quality response must simultaneously satisfy multiple critical features to achieve a high evaluation score. On the other hand, refinements are path-dependent: the improvement space of subsequent solutions is largely determined by the direction of early-stage attempts. Although this is perhaps the nature of LLM-based generators, whose output depends primarily on historical context, it inevitably creates a “Matthew Effect” along the refinement trajectory, as early progress in one dimension attracts further refinement to that same dimension, starving other dimensions and trapping the search around local optima.

###### Definition .2 (Multidimensional Problem Refinement Trajectory) .

Consider an open-ended problem parameterized by ( D , λ , β ) (D,\lambda,\beta) . The solution space is 𝒴 = ℕ D \mathcal{Y}=\mathbb{N}^{D} , where D ∈ ℕ + D\in\mathbb{N}^{+} represents the number of dimensions. The score of a solution y y is given by V ⁡ ( y ) = 1 − λ min d = 1 D ​ y d , \displaystyle V(y)=1-\lambda^{\min_{d=1}^{D}y_{d}}, (8) where λ ∈ ( 0 , 1 ) \lambda\in(0,1) represents the refinement strength. The policy is modeled as: 1. The initial solution y ⁡ ( 0 ) = [ 0 , ⋯ , 0 ] D y(0)=[0,\cdots,0]^{D} .

2. At each step t t , exactly one dimension d d will be selected to be refined, with p d ​ ( t ) = Prob ​ ( refine dimension ​ d ​ at step ​ t ) = 1 + β ⋅ y d ​ ( t − 1 ) D + β ⋅ ( t − 1 ) , \displaystyle p_{d}(t)=\text{Prob}(\text{refine dimension }d\text{ at step }t)=\frac{1+\beta\cdot y_{d}(t-1)}{D+\beta\cdot(t-1)}, where β \beta captures the extent to which the refinement is biased towards existing attempts.

3. Denote d ⁡ ( t ) d(t) the selected dimension to refine, the improved solution is y ⁡ ( t ) = { y d ​ ( t − 1 ) + 1 d = d ⁡ ( t ) y d ​ ( t − 1 ) o . w . . y(t)=\begin{cases}y_{d}(t-1)+1&d=d(t)\\ y_{d}(t-1)&o.w.\end{cases}.

Seeking to rigorously capture this mismatch, we introduce Definition .2 , the multidimensional problem refinement trajectory that mathematically formalizes the nature of open-ended problems and path-dependent policies. This model accurately reflects the problems of interest. First, the score function V ⁡ ( y ) = 1 − λ min d ⁡ y d V(y)=1-\lambda^{\min_{d}y_{d}} reflects the bottleneck principle : overall quality is limited by the weakest dimension d ∈ [ D ] d\in[D] , analogous to how a scientific solution must satisfy multiple criteria (correctness, efficiency, generality). Second, the parameter β \beta controls the strength of path dependence. When β = 0 \beta=0 , each dimension is equally likely to be refined at each step, corresponding to a uniform exploration; as β → ∞ \beta\to\infty , refinement concentrates on the first explored dimension, resulting in pure exploitation. In practice, LLM-based refinement lies between these extremes: models tend to elaborate on existing ideas rather than introduce orthogonal improvements, corresponding to moderate β \beta . Lastly, this trajectory model makes an elegant assumption that refinement always occurs, simplifying the nuance of the real-world generator G G , which can generate a solution y ⁡ ( t ) y(t) worse than existing results.

##### Number of global independent trials C C .

To overcome the above limitation, existing work [ 73 , 23 , 82 ] has introduced various approaches, such as the “island” within which the policy refines solutions locally and exchanges them periodically. However, what are the underlying mechanisms in effect? The following theorem argues that, the structural benefit of these complicated systems is primarily derived from a single fundamental factor: the number of independent experiments, i.e., the number of trajectories C C .

###### Theorem .3 .

Consider an open-ended refinement problem parameterized by ( D , λ , β ) (D,\lambda,\beta) with D ≥ 2 D\geq 2 and β > 0 \beta>0 . Assume the total budget N N is split into C C independent trajectories, each performing L = N C L=\frac{N}{C} refinement steps.

Then, as s → 1 s\to 1 , the optimal allocation ( L ⋆ , C ⋆ ) (L^{\star},C^{\star}) that minimizes the total budget N N subject to the reliability constraint P fail ​ ( L , C ) ≤ ϵ P_{\text{fail}}(L,C)\leq\epsilon satisfies L ⋆ = Θ ⁡ ( log λ ⁡ ( 1 − s ) ) , C ⋆ = Θ ⁡ ( log ⁡ 1 ϵ ) . \displaystyle L^{\star}=\Theta\!\left(\log_{\lambda}(1-s)\right),\qquad C^{\star}=\Theta\!\left(\log\frac{1}{\epsilon}\right). (9) Here Θ ⁡ ( ⋅ ) \Theta(\cdot) hides the dependency on D D and β \beta .

###### Proof.

Let r = ⌈ log λ ⁡ ( 1 − s ) ⌉ . \displaystyle r=\left\lceil\log_{\lambda}(1-s)\right\rceil. (10) Since 0 < λ < 1 0<\lambda<1 , a trajectory reaches score at least s s after L L refinement steps if and only if min d ∈ [ D ] ⁡ y d ​ ( L ) ≥ r . \displaystyle\min_{d\in[D]}y_{d}(L)\geq r. (11) Thus r r is the number of successful refinements required in every dimension, and r → ∞ r\to\infty as s → 1 s\to 1 .