Structured Scaling of AI Discovery Across Diverse Scientific Domains — part 11/12

Cleaned-text code points [156284, 171691). Bibliography omitted here; source.md includes it.

ZAPBench evaluates neural-activity forecasting at whole-brain, single-neuron resolution. Structural maps of neural wiring are essential for understanding circuit organization, but they do not by themselves specify the time-varying circuit dynamics that transform sensory input and internal state into neural responses. A functional account therefore requires predicting how activity evolves across the brain over time. ZAPBench provides a quantitative benchmark for this problem by asking models to forecast the activities of all recorded neurons from a short temporal context, enabling direct comparison of forecasting models and analysis of how whole-brain dynamics unfold across stimulus conditions. It is built from a two-hour light-sheet fluorescence recording of a larval zebrafish that was exposed to a sequence of visual stimuli designed to elicit a range of behaviors; the raw volumetric video was aligned, motion-stabilized, and segmented into activity traces, yielding a data matrix of 71,721 71{,}721 neurons over 7,879 7{,}879 time steps (a sampling interval of roughly 914 914 ms per step) spanning nine stimulus conditions ( gain , dots , flash , taxis , turning , position , open loop , rotation , and dark ).

##### Forecasting task and data splits.

The task is to predict the joint future activity of all 71,721 71{,}721 neurons up to 32 32 steps ahead given only 4 4 steps of past context—a high-dimensional multivariate forecasting problem. Because the traces are derived from the volumetric recording, a forecaster may operate either on the extracted per-neuron traces or on the 3D volumes directly: the benchmark’s strongest hand-designed baseline is a volumetric U-Net that predicts the brain video, exploiting spatial structure that is lost when the volumes are reduced to traces, but at a much higher computational cost—the video model is reported to train in roughly 36 36 hours on 16 16 A100 GPUs, versus under four hours on a single 16GB GPU for trace-based forecasters. Following ZAPBench, each stimulus condition is split 70 / 10 / 20 70/10/20 into contiguous train, validation, and test segments.

##### Evaluation and search protocol.

Candidate programs are scored by the mean absolute error (MAE) between predicted and recorded activity, averaged over all neurons and over the full 32 32 -step (or 1 1 , 4 4 , 8 8 , 16 16 steps in different tasks) horizon. The validation split serves two purposes: it drives model selection including hyperparameter choices and early stopping; and it supplies the single scalar (validation MAE) that guides the search. The test split is never seen during search or model selection. SimpleTES is initialized from one neural starter program, a shared-weight per-neuron forecaster (Fig. 4 f), and edits the body of its forecast routine. Each candidate is trained and scored in an isolated evaluator.

### M Single-Cell RNA-Seq Denoising

Single-cell RNA sequencing (scRNA-seq) resolves gene expression in individual cells [ 89 , 173 , 87 ] , but its measurements are corrupted by low capture efficiency and stochastic dropout [ 146 ] , motivating dedicated denoising algorithms [ 147 ] . Denoising has no ground truth to check against; quality is assessed by the molecular cross-validation framework of [ 11 ] , which partitions observed UMI counts by binomial subsampling and underpins the OpenProblems benchmark [ 86 ] . The core difficulty is a tension between two objectives: mean-squared error in log-normalized space rewards faithful recovery of relative expression, while the Poisson negative log-likelihood rewards count consistency after library-size rescaling, and aggressively optimizing one degrades the other.

We follow the molecular cross-validation protocol [ 11 ] , splitting the Pancreas dataset into a training matrix the algorithm observes and a held-out test matrix it does not. Candidates are scored on two normalized metrics, log-space MSE and Poisson negative log-likelihood, with the Poisson term acting as a hard constraint that rejects any solution failing it, and the final score is the mean of the two. Following [ 165 ] , we initialize SimpleTES with MAGIC [ 147 ] , cap each candidate at 400 seconds, and run evolution exclusively on Pancreas; PBMC and Tabula Muris Senis Lung are withheld entirely and used only for final evaluation, so the reported numbers measure cross-dataset generalization to held-out datasets.

Across the held-out datasets in Table 15 , SimpleTES reaches a Tabula score of 0.74 0.74 , surpassing the prior state-of-the-art method TTT-Discover [ 165 ] ( 0.73 0.73 ) and matching it on PBMC ( 0.71 0.71 ), while outperforming MAGIC [ 147 ] , ALRA [ 81 ] , and every other baseline on both. Because evolution never saw PBMC or Tabula, these gains are direct evidence that the discovered algorithm generalizes rather than fitting the search distribution.

Inspection shows that SimpleTES departs structurally from the baselines. Where MAGIC builds a single diffusion operator and TTT-Discover elaborates that template with variance-stabilizing transform ensembling and low-rank refinement, the discovered program constructs several independent denoisers, including multi-scale diffusion under both correlation- and Euclidean-based graph operators, log-space diffusion, neighbor averaging, PCA imputation, and NMF reconstruction, and scores each internally on the training split alone. It then blends them into a data-driven ensemble, weighting candidates by their inverse Poisson and MSE losses under the Poisson constraint, and applies a gene-mean calibration that clips per-gene scaling near 1.0 1.0 to preserve count fidelity. None of these choices are dataset-specific: the ensemble weights are fit from the training split at runtime, which is what lets the same program generalize to the unseen PBMC and Tabula distributions.

### N Lasso Regularization Path

##### Overview.

The lasso regularization path is a core computational primitive in high-dimensional statistics, arising naturally in cross-validation and model selection across domains from genomics to finance. Computing the full path of solutions across a grid of regularization values is orders of magnitude more efficient than solving each problem independently, but demands careful algorithmic design to exploit warm starts and sparsity structure. The de facto standard solver, glmnet [ 39 ] , represents decades of expert engineering. This task asks whether SimpleTES can discover a solver that is faster than glmnet while maintaining the same float64 precision and correctness guarantees.

##### Experimental setting.

Each candidate is a self-contained C++ program that reads a binary-encoded problem from standard input and writes the complete coefficient path to standard output. Correctness is evaluated on a separate fresh problem that is not used for timing, preventing caching or test-set overfitting. The surrogate metric optimized during evolution is the geometric mean of solve times over 17 synthetic problem sizes. These instances span the axes that govern lasso-path performance: sample-to-feature ratio n / p n/p , design-matrix sparsity, active-set density, and feature correlation from near-independent to highly correlated Toeplitz structure.

This evaluation surface is deliberately heterogeneous: a solver specialized for wide problems with p ≫ n p\gg n can fail on tall problems with n ≫ p n\gg p , and vice versa. The 11 real-world datasets in Table 16 are not used during evolution and therefore test whether the discovered solver generalizes beyond the synthetic search distribution. We initialize SimpleTES from a faithful C++ port of glmnet ’s Gaussian lasso path, including the covariance update for p < 500 p<500 , the naive residual update for p ≥ 500 p\geq 500 , warm starts, sequential strong-rule screening [ 142 ] , active-set inner loops, and KKT verification.

##### Results analysis.

All reported solutions pass the held-out correctness check. Across the 11 real-world datasets, SimpleTES achieves an average speedup of 2.17 × 2.17\times over glmnet and 14.08 × 14.08\times over sklearn . The gains are largest in tall regimes: on DNA ( 1700 × 180 1700\times 180 ), SimpleTES is 9.56 × 9.56\times faster than glmnet . It also improves on the large sparse RCV1 design by 1.76 × 1.76\times over glmnet and 5.49 × 5.49\times over sklearn ; the gains against sklearn are particularly large on high-dimensional biological datasets, including 50.77 × 50.77\times on TCGA BRCA and 10.16 × 10.16\times on Duke Breast Cancer.

##### Case study.

The initialization uses coordinate descent throughout: the covariance method for p < 500 p<500 and the naive residual method for p ≥ 500 p\geq 500 . The discovered program instead uses a geometry-aware switching rule. For moderate-dimensional, tall designs ( p ≤ 2000 p\leq 2000 and n ≥ p / 4 n\geq p/4 ), it replaces coordinate descent with an exact LARS homotopy solver that traces the regularization path analytically and updates the active set at each kink with rank-one inverse-Gram updates [ 34 ] . For wide or sparse designs, it retains coordinate descent with strong-rule screening, an active-set inner loop, and KKT verification.

### O Mathematical Scientific Discovery Tasks

Extremal analysis and combinatorial construction ask for explicit mathematical solutions that optimize precisely defined quantities: a distribution, a witness function, an integer set, a geometric packing, or a sign matrix. These tasks are a natural setting for scientific discovery because progress depends on finding the right structural solution, and each candidate can be checked by a deterministic evaluator rather than by a noisy empirical proxy.

#### O.1 Extremal analysis

Extremal analysis tasks search over constrained functions and optimize overlap or autoconvolution functionals. Candidate solutions are represented as discretized step functions or signed piecewise-constant witnesses, with feasibility enforced by support, range, and unit-mass constraints.

##### Erdős minimum-overlap problem.

Erdős’ minimum-overlap problem asks how to distribute a unit amount of mass over [ 0 , 2 ] [0,2] so that the distribution has as little one-sided overlap as possible with the complements of its shifted copies.

The evaluator reports the translated-overlap objective through 1 / Ψ ⁡ ( h ) 1/\Psi(h) during search, while the standard value Ψ ⁡ ( h ) \Psi(h) is reported for comparison.

Best result and discovered solution. The discovered witness is near-binary: most grid cells are driven close to either zero or one, rather than remaining near the uniform initialization. This structure arranges the unit mass so that the translated-overlap profile is flattened across the high-risk shift range, reducing the worst shift rather than only improving the average shift behavior. The solution is therefore interpretable as an equioscillation-style witness: the best solution balances multiple competing translated-overlap constraints instead of optimizing a single shift locally ( Figure 10 a).

Case Analysis. The strongest program uses a coarse-to-fine optimization pipeline. It first searches over coarse discretizations to identify promising mass layouts, then refines them with constrained local optimization and projected polishing steps that preserve the exact discrete mass constraint and the box constraint 0 ≤ h ≤ 1 0\leq h\leq 1 . This code structure is important because naive local perturbations around the uniform witness tend to improve only a small subset of shifts, whereas coarse-to-fine refinement can reorganize the global mass pattern before polishing the final overlap profile.

##### Autocorrelation Inequalities.

The First, Second, and Third Autocorrelation Inequalities form a family of extremal autoconvolution problems.

During search, the evaluators cast the tasks as maximization by using 1 / Φ 1 1/\Phi_{1} , Φ 2 \Phi_{2} , and 1 / Φ 3 1/\Phi_{3} , while the standard objectives are reported in the result tables.

Best results and discovered solutions. The three autocorrelation tasks lead to different witness structures. For AC1, the strongest discovered witness concentrates mass near the support boundary, reducing the peak of the non-negative autoconvolution but still trailing the leading public result. For AC2, the best witness is sparse and produces a long near-flat plateau in the autoconvolution profile, a structure that improves the norm ratio by avoiding a narrow peak. For AC3, the signed witness becomes oscillatory and uses cancellation to control the largest absolute autoconvolution value. These solutions show that the same evaluation-driven loop can search across non-negative and signed witness classes without a task-specific symbolic derivation ( Figure 10 b–d).

Case Analysis. The best programs implement different search strategies for the three objectives. AC1 uses a simplex-projected mass-transfer search over a 1024-bin witness, repeatedly moving mass while preserving non-negativity and unit mass. AC2 combines FFT-based convolution evaluation with L-BFGS-B refinement, making it efficient to evaluate many candidate witnesses and polish sparse structures. AC3 uses a discrete cosine transform parameterization that directly searches signed oscillatory patterns and optimizes the cancellation structure of f ∗ f f\ast f . The code-level adaptation is therefore objective-specific even though the outer SimpleTES procedure is unchanged.

#### O.2 Combinatorial construction

Combinatorial construction tasks search over finite structured solutions: integer sets, unequal-circle packings, and { − 1 , 1 } \{-1,1\} sign matrices. Their evaluators are exact or feasibility-based, using integer arithmetic for sumsets, geometric non-overlap checks for packings, and exact determinant computation for sign matrices.

##### Sum–Difference Problem.

The Sum–Difference Problem comes from classical additive combinatorics and is closely related to more-sums-than-differences sets and Ruzsa-type inequalities [ 92 , 54 , 116 , 117 ] .

Candidate sets contain at most 512 integers with elements bounded in [ − 10 6 , 10 6 ] [-10^{6},10^{6}] . The evaluator computes | A + A | |A+A| and | A − A | |A-A| exactly using integer arithmetic.

Best results and discovered solutions. The discovered set has a highly regular arithmetic backbone rather than an irregular collection of integers. Most consecutive gaps follow a long progression pattern, while a small number of fringe positions receive local corrections. These sparse perturbations enlarge the sumset more efficiently than the difference set, producing a construction that is both high-scoring and structurally interpretable ( Figure 10 e).

Case Analysis. The strongest program maintains exact sum and difference multiplicity tables during search. It alternates aggressive pruning, greedy additions, and local replacements, allowing each candidate edit to be scored by its exact effect on the two set expansions. This is more efficient than repeatedly rebuilding A + A A+A and A − A A-A from scratch, and it makes the search sensitive to local fringe edits that would be difficult to identify from the final scalar score alone.

##### Circle Packing in a Unit Square.

Unequal-circle packing in a square is a classical problem in continuous optimization and discrete geometry [ 105 , 136 , 55 ] .

Invalid configurations receive zero score.