Towards Execution-Grounded Automated AI Research — part 1/4

Cleaned-text code points [0, 15323). Bibliography omitted here; source.md includes it.

# Towards Execution-Grounded Automated AI Research

###### Abstract

Automated AI research holds great potential to accelerate scientific discovery. However, current LLMs often generate plausible-looking but ineffective ideas. Execution grounding may help, but it is unclear whether automated execution is feasible and whether LLMs can learn from the execution feedback. To investigate these, we first build an automated executor to implement ideas and launch large-scale parallel GPU experiments to verify their effectiveness. We then convert two realistic research problems – LLM pre-training and post-training – into execution environments and demonstrate that our automated executor can implement a large fraction of the ideas sampled from frontier LLMs. We analyze two methods to learn from the execution feedback: evolutionary search and reinforcement learning. Execution-guided evolutionary search is sample-efficient: it finds a method that significantly outperforms the GRPO baseline (69.4% vs 48.0%) on post-training, and finds a pre-training recipe that outperforms the nanoGPT baseline (19.7 minutes vs 35.9 minutes) on pre-training, all within just ten search epochs. Frontier LLMs often generate meaningful algorithmic ideas during search, but they tend to saturate early and only occasionally exhibit scaling trends. Reinforcement learning from execution reward, on the other hand, suffers from mode collapse. It successfully improves the average reward of the ideator model but not the upper-bound, due to models converging on simple ideas. We thoroughly analyze the executed ideas and training dynamics to facilitate future efforts towards execution-grounded automated AI research.

###### Keywords:

## 1 Introduction

We envision automated AI research: LLMs generate research ideas to tackle important research problems, implement the ideas as code, run experiments to verify the effectiveness, and continuously learn from the execution results. If successful, these automated AI researchers could automatically develop and identify effective research ideas in a massive search space, thereby scalably converting compute into scientific discovery; the discovered ideas could, in turn, improve frontier AI models themselves, enabling recursive self-improvement. Despite the promise, automated AI research is bottlenecked by the ability of LLMs to generate effective ideas. Si et al. (2025b) and Si et al. (2025a) evaluated the quality of LLM-generated research ideas through large-scale expert review and found that LLM ideas often look convincing but are ineffective after being executed by human researchers.

This highlights the need to ground idea generation in execution. However, obtaining execution results of ideas in an automated and scalable manner is challenging, especially since we are targeting open-ended AI research where any ideas expressible in natural language are within our action space. To tackle this, we design and build a high-throughput automated idea executor that can implement hundreds of model-generated ideas and execute them in parallel to obtain the experiment results as execution feedback.

To study the extent to which we can automate realistic LLM research, we chose two GPU-intensive research problems (LLM pre-training and post-training) that are critical for improving the capabilities of LLMs as the research environments for our automated AI researchers. For the first time, we demonstrate that our automated executor can implement a large fraction of LLM-generated ideas on these challenging open-ended research problems, with over 90% execution rates on the pre-training environment with Claude-4.5-Sonnet and Claude-4.5-Opus.

To analyze whether grounding on execution-feedback can improve LLM idea generation, we define objective performance metrics for both environments and analyze the strengths and weaknesses of two popular learning algorithms: evolutionary search and reinforcement learning.

We use our automated executor to guide evolutionary search. Within ten search epochs, this execution-guided search finds a post-training recipe that outperforms the GRPO baseline (69.4% vs 48.0%) on the task of post-training a 1.5B model for math reasoning, and a pre-training recipe that outperforms the nanoGPT baseline (19.7 minutes vs 35.9 minutes) on the task of minimizing the training wall-clock time to reach the target validation loss (Table 1 ). Our analysis shows that models are often generating algorithmic ideas apart from tuning hyper-parameters, and evolutionary search significantly outperforms best-of-N under the same sampling budget. However, when analyzing the scaling trend, only Claude-4.5-Opus shows a clear scaling curve, while both Claude-4.5-Sonnet and GPT-5 tend to saturate early.

We then use the automated executor as the reward function in an RL loop to finetune Qwen3-30B. We show that RL with execution reward can successfully improve the average reward of the ideator model, similar to typical RL from verifiable rewards. However, RL does not improve the max reward, which is the more important metric for scientific discovery. In fact, we reveal that RL causes the ideator model to converge on a few easy-to-implement ideas, resulting in a collapse in thinking length and idea diversity.

In summary, we develop a large-scale automated idea executor system that can implement research ideas for open-ended and realistic research problems. Using this automated executor, we conduct an in-depth analysis of how well LLM ideators can learn from execution feedback to improve effectiveness through evolutionary search and RL. Execution-guided evolutionary search is sample-efficient and effective, but shows limited scaling. RL from execution reward suffers from diversity collapse and does not improve the upper-bound. We additionally provide extensive analysis on the executed ideas and suggest promising directions to improve the existing learning algorithms. Altogether, we demonstrate the feasibility and potential of grounding LLM ideation in automated execution and uncover important limitations for future improvement.

## 2 Automated Idea Executor

To measure the effectiveness of model-generated ideas, we build an automated executor that takes natural language research ideas as input, generates code implementations, runs the experiments on the backend, and returns the idea’s benchmark performance as the final output.

### 2.1 Research Environments for Ideation

Our automated idea executor is grounded in specific research environments, where each environment consists of a research problem, a baseline codebase, a benchmark to measure performance on, fixed training and evaluation data, and evaluation metrics. When constructing the research environments, we aim to select research problems that are open-ended, so that there is ample room for new algorithmic innovations, and at the same time have well-established baselines and benchmarking metrics so that measuring effectiveness is straightforward. In this work, we construct both a pre-training environment and a post-training environment for the automated AI researchers to work on. 1 1 1 We open-source our environments and idea execution trajectories in https://github.com/NoviScl/Automated-AI-Researcher .

Pre-Training Task: Improving nanoGPT In the nanoGPT environment, we provide a baseline codebase adapted from the nanoGPT speedrun ( Jordan et al., 2024 ) and ask the ideator model to brainstorm possible improvements. The original speedrun task is to minimize the time to pre-train a 124M GPT-2 model ( Radford et al., 2019 ) on the FineWeb corpus ( Penedo et al., 2024 ) to reach a validation loss of 3.28 on the validation set on 8 H100 GPUs. We did several modifications to the original speedrun setting. First, we introduce a proxy reward equal to the reciprocal of the validation loss ( 1 l ​ o ​ s ​ s \frac{1}{loss} ) when performing the search and RL experiments in later sections of the paper. This way, we can fix the training wall-clock time to be 25 minutes and ask the model to directly optimize the proxy reward under this fixed budget, so that we can avoid different runs having vastly different runtimes. We report the validation loss or the proxy reward metric in most plots, and only measure and report the training time metric for the top solution in order to directly compare it with the human experts’ solutions on the original nanoGPT speedrun leaderboard. Second, to avoid any possible reward hacking, we freeze all evaluation hyper-parameters and implement an inference function that predicts one future token at a time to prevent models from changing the attention mechanism in a way that leaks future tokens (which happened multiple times during our initial development). We use this inference function during the final validation after each training run.

Post-Training Task: Improving GRPO In the GRPO environment, the baseline is an implementation of the GRPO algorithm ( Shao et al., 2024 ) that finetunes a Qwen2.5-Math-1.5B checkpoint ( Yang et al., 2024 ) on the MATH dataset ( Hendrycks et al., 2021 ) . The ideator model needs to brainstorm post-training algorithms that are more effective than the baseline. We specify a fixed training wall-clock time budget and use the max accuracy on the MATH validation set during training as the metric. To prevent reward hacking, we keep all validation-related code in a separate file and do not allow the automatic executor to access or modify it.

In both environments, we do not set any constraints on the ideation scope, so anything between extensive hyperparameter tuning and novel model architecture or training algorithms is within scope.

### 2.2 System design

The automated idea executor can be viewed as a high-level API whose input is a batch of natural language ideas, and the output is the benchmark performance of each idea. There are three core building blocks of this API (Figure 1 ): Implementer – the server that generates the code diff for the idea and applies those changes; Scheduler – a middle layer that receives the list of codebases and allocates resources to run experiments; Worker – the cluster with GPU available that runs the experiments and uploads the experiment results.

#### Implementer

The implementer is hosted on a CPU machine with high IO capacity. First, the user submits a batch of natural language ideas. Then, for each idea, the implementer makes parallelized API calls to the code execution LLM to obtain a diff file that can be patched into the corresponding baseline codebase. To optimize for efficiency, we prompt the code execution LLM with both the idea and the baseline codebase to sample 10 10 code diff files in parallel. For each sample, if the generated diff file cannot be patched into the original codebase, we provide the patch log and ask the model to revise the original generation. We repeat this sequential self-revision for a maximum of 2 2 times. In the end, we return the first code diff file that can be successfully patched into the baseline codebase. The patched codebase is then submitted to a cloud bucket as a .zip file.

#### Scheduler

Under a set clock frequency, the scheduler downloads the new codebases from the cloud. If the codebase has not been executed, the scheduler examines the resource requirement of the given research environment and prepares a job configuration to be submitted.

#### Worker

Once the scheduler finds available resources, it connects the prepared job configuration with the GPU resource and initializes the worker to run the experiment. If the execution of the experiment is successful, the worker will upload the experiment logs including all performance metrics to another cloud bucket ( wandb ) along with the complete metadata: idea content, code change, execution log, etc. If the execution fails (e.g., due to bugs in code implementation), the worker halts. The user (i.e., the ideator model) can then download the execution results and see the performance of the batch of ideas they submitted with full training logs.

## 3 Benchmarking LLM Ideators and Executors

The prerequisite for an execution-grounded feedback loop is that current LLMs can serve as both ideators and executors, so that we can get meaningful reward signals for the models to learn from. To examine this prerequisite, we first benchmark various frontier LLMs as both the ideator and the executor.

### 3.1 End-to-End Ideation and Execution

In the first setting, we sample ideas from an LLM, and use the same LLM as the code execution model to execute its own ideas. We sample and execute 50 ideas from Claude-4.5-Opus, Claude-4.5-Sonnet, and GPT-5, and measure several metrics: (1) completion rate: the percentage of ideas that are successfully executed with a valid (non-zero) experiment result after execution; (2) average performance: the average validation accuracy or loss for all the successfully executed ideas among the 50 samples; (3) best performance: the highest validation accuracy or lowest validation loss among all executed ideas. We present results in the top row of Figure 2 . Notably, a large fraction of the sampled ideas can indeed be executed successfully, with Claude-4.5-Opus and Claude-4.5-Sonnet having a significantly higher execution rate than GPT-5. Moreover, the best-of-N performance ( N = 50 N=50 ) of these models can already beat the original baseline solutions. For example, on the GRPO environment, Claude-4.5-Sonnet gets a max accuracy of 60.4% as compared to the baseline of 48.0%; on nanoGPT, Claude-4.5-Opus gets a lowest loss of 3.237 as compared to the baseline of 3.255.

### 3.2 Comparing Ideators with the Same Executor

In the second setting, we fix the executor model to be GPT-5 and use different ideator models to sample ideas. As shown in the bottom row of Figure 2 , even when the ideator and executor are different models, the execution rate is still decent (ranging from 42% to 78%), although we do notice that the same ideas from Claude-4.5-Sonnet get a lower execution rate when executed by GPT-5 instead of itself (84% vs 42% on GRPO and 90% vs 78% on nanoGPT). Moreover, frontier open-weight models like Kimi-K2-Thinking ( Kimi Team, 2025 ) and Qwen3-235B-A22B ( Yang et al., 2025a ) can also get non-trivial completion rates and achieve best-of-N performance that outperforms the baseline solutions in this setting. For example, Qwen3-235B achieves a max accuracy of 50.2% on GRPO and min loss of 3.238 on nanoGPT with N = 50 N=50 , both better than the baselines.

These benchmarking results demonstrate the feasibility of the automated ideation and execution loop. Next, we build search scaffolds and RL training loops to examine whether models can learn from the execution feedback.

## 4 Execution-Guided Evolutionary Search

Evolutionary search ( Koza, 1994 ; Lehman et al., 2023 ) is a traditional optimization method without the need for gradient updates. We develop an evolutionary search scaffold on top of frontier LLMs to optimize for effective ideas based on execution feedback. We introduce our search method that blends exploration and exploitation, its effectiveness on our two research environments, and various analyses of the generated ideas throughout the evolutionary search process.

### 4.1 Search Scaffold