Self-Improvements in Modern Agentic Systems: A Survey — part 13/14

Cleaned-text code points [184194, 200018). Bibliography omitted here; source.md includes it.

Account for resource efficiency and supervision. While trajectory tracking bounds the evaluation process, the exact composition of the cumulative cost ( b t b_{t} ) dictates the real-world practicality of the improvement pipeline. Self-improvement inherently consumes compute, API tokens, wall-clock time, etc. A rigorous protocol should provide a transparent cost breakdown rather than just final task success to ensure fair comparisons of efficiency across methods ( Lange et al., 2026 ; Ni et al., 2025 ) . Crucially, any reliance on external human oversight fundamentally compromises the “self” in self-improvement. Therefore, if human-in-the-loop interventions are employed, their exact magnitude and modality should be explicitly quantified, as varying levels of external supervision fundamentally alter the agent’s autonomous learning dynamics ( Wang et al., 2024c ; Drouin et al., 2024b ) .

Track stability, regressions, and safety over time. Iterative self-modification can induce goal drift, reward hacking, and compounding errors in memory or tool use ( Amodei et al., 2016 ; Bai et al., 2022 ; Ruan et al., 2024a ; Yang et al., 2025c ) . Consequently, evaluation should track regression rates on previously solved tasks, tail-risk indicators, and safety policy violations across iterations, rather than relying exclusively on mean success rates ( Levy et al., 2026 ; Yin et al., 2025a ) .

Recommended reporting items. For each method and benchmark suite, empirical reports should consistently provide: initial baseline performance, performance after a fixed improvement budget, learning curves across iterations, transfer capabilities on held-out tasks, regression rates on previously solved instances, and a comprehensive cost summary (compute, tool invocations, time, and human input).

#### 8.1.2 Judge-Based Measurement

A substantial fraction of agent workloads lack formal success criteria, rendering deterministic checks ( Φ metric \Phi_{\text{metric}} ) inapplicable. In open-ended or partially observable domains, evaluation necessitates parameterized judges ( Φ judge \Phi_{\text{judge}} ) to translate complex long-horizon trajectories and intermediate artifacts into structured scores. These evaluators utilize standard LLMs or autonomous Agent-as-a-Judge pipelines ( Zhuge et al., 2024b ; You et al., 2026 ; Zhang et al., 2025b ; Wadhwa et al., 2025 ; Xu et al., 2025d ; Han et al., 2025 ) . However, because parameterized judges only approximate human-aligned ground-truth objectives, they may introduce distinct vulnerabilities—most notably, the risk of a self-improving agent over-optimizing to the judge’s latent biases. Consequently, judge-based evaluation demands specific operational safeguards.

##### Specify the judge and the judging budget.

When Φ judge \Phi_{\text{judge}} is employed to compute the evaluation score ( m t m_{t} ), the protocol should mandate full transparency regarding the judge’s identity and parameters ( θ judge \theta_{\text{judge}} ). This includes the exact foundation model version, the prompt formulation, the precise rubric ( κ \kappa ), and the environmental evidence exposed to the judge. Crucially, the protocol should isolate the judging budget from the agent’s execution budget ( b t b_{t} ). Because evaluation reliability fluctuates heavily with the resources allocated to the judge—such as context window limits, allowed multi-agent debates, or tool-use steps—failing to report the judge’s computational overhead obscures whether the agent genuinely improved or the evaluator merely became more exhaustive.

##### Prevent over-optimization to the judge and report reliability signals.

If the identical evaluation operator Φ judge \Phi_{\text{judge}} is used both to drive updates and to report final results, the self-improving system will likely over-optimize to the judge’s latent biases rather than the intended objective. To prevent this, rigorous protocols should enforce evaluator independence by employing a distinct judge configuration for final reporting, such as a more capable foundation model ( θ judge ′ \theta_{\text{judge}}^{\prime} ) or an orthogonal evaluation rubric ( κ ′ \kappa^{\prime} ). Reliability can be supported by empirical evidence, such as repeated runs with variance estimates, aggregation across multiple judge instances, and calibration against a verifiable subset using Φ metric \Phi_{\text{metric}} or targeted human review.

### 8.2 Benchmarking Improvement

Benchmarks for self-improving agents vary along two attributes that determine what an evaluation can validate. The first attribute is the update channel. Some methods update the foundation model ( θ \theta ), while others keep the foundation model fixed and update only the operational scaffolding ( Σ \Sigma ). The second attribute is the evaluation interface. Some benchmarks use single-shot input-output evaluation, while others require interactive multi-step behavior with tools, memory, and environment feedback. These attributes are complementary, and they jointly determine the appropriate transfer tests, regression checks, and cost accounting. This section distinguishes mechanism-focused benchmarks, which isolate how improvement is produced, from domain-focused benchmarks, which instantiate improvement under realistic task and feedback constraints.

#### 8.2.1 Mechanism Benchmarks

Mechanism-focused benchmarks serve as the empirical testbeds for the rigorous evaluation protocols established above. Rather than assessing static zero-shot execution, these environments isolate specific update channels—such as memory expansion or prompt refinement—to quantify continuous adaptation. By enforcing strict resource budgets ( b t b_{t} ) and evaluating on entirely held-out task splits ( 𝒟 eval \mathcal{D}_{\text{eval}} ), they systematically distinguish genuine capability transfer from localized overfitting. Figure 12 summarizes representative benchmarks and their usage across self-improving agent systems, grouped by evaluation interface.

##### FM -Level Evaluation.

FM -level benchmarks isolate capability gains derived from updating foundation model parameters ( θ t → θ t + 1 \theta_{t}\rightarrow\theta_{t+1} ) while keeping the scaffolding constant. Because parametric updates are susceptible to catastrophic forgetting and subtle training-data leakage, testbeds in this category prioritize rigorous provenance tracking and strict dataset compartmentalization ( Aleithan et al., 2024 ) . Rather than merely measuring peak zero-shot accuracy, representative suites enforce retention auditing across adjacent task families to expose distributional drift ( Ruan et al., 2024a ) . Executable and repository-based evaluations are are favored here because they provide objective checking ( Φ metric \Phi_{\text{metric}} ) and fine-grained regression detection ( Ni et al., 2025 ) .

##### Scaffold -Level Evaluation.

When the backbone model is frozen ( θ t + 1 = θ t \theta_{t+1}=\theta_{t} ), improvements arise from changes to the scaffolding ( Σ t → Σ t + 1 \Sigma_{t}\rightarrow\Sigma_{t+1} ), such as prompt policies, critique loops, memory write and retrieval rules, and tool routing. Mechanism-focused evaluation for this channel benefits from component isolation. The protocol should specify which architectural component is modified and evaluate both capability gains and new failure surfaces. Prompt-policy changes should be evaluated under paraphrases, formatting shifts, and longer contexts to measure genuine robustness rather than repeated optimization on a single template. Memory changes should be evaluated with long-horizon recall and cross-modal or multi-party consistency, as well as resistance to poisoning, unauthorized disclosure, active-forgetting failures, and compounding errors ( Yang et al., 2025c ; Zhu et al., 2026 ; Ren et al., 2026 ) . Tool changes should be evaluated with verifiable benchmarks that measure tool invocation, tool selection, and argument grounding under multi-step interaction, since common failures arise from ordering mistakes and near-miss parameterization rather than incorrect intent ( Patil et al., 2025 ; Huang et al., 2024b ; Shen et al., 2024 ; Wang et al., 2024c ) .

##### Attribution Across Mechanisms.

Mechanism comparisons require precise attribution. Evaluations should include ablations that change the updated component, replay-style rollouts that hold environments fixed while swapping the updated module, and regression checks on tasks that were previously solved. Even partial attribution improves interpretability by separating true architectural gains from reasoning variance, retrieval noise, or mere benchmark exploitation.

#### 8.2.2 Domain Benchmarks

While mechanism benchmarks isolate specific update channels, domain-focused evaluation instantiates the evaluation framework under realistic feedback and task distributions. For each domain, a rigorous protocol should state the available feedback signal, the permitted update channel ( θ \theta or Σ \Sigma ), the granted resource budget ( b t b_{t} ), and the strictly held-out sets ( 𝒟 eval \mathcal{D}_{\text{eval}} ) used for transfer and regression checking. Across domains, it is standard practice to report learning curves under fixed budgets, transfer capabilities to held-out task families, and track regression and safety indicators alongside mean success rates ( Xi et al., 2024 ; Levy et al., 2026 ; Yin et al., 2025a ) .

##### Software Engineering.

Software engineering environments supply deterministic, executable feedback ( Φ metric \Phi_{\text{metric}} ), making them ideal testbeds for grounding self-improvement claims. Rather than relying on abstract reporting rules, modern suites in this domain structurally enforce rigorous evaluation: they quantify capability transfer across completely disjoint repositories and utilize extensive, hidden regression tests to expose localized overfitting. Representative platforms range from repository-level issue resolution, such as SWE-bench and its leakage-robust variants ( Jimenez et al., 2024 ; Aleithan et al., 2024 ) , to interactive codebases that stress continuous exploration ( Qiu et al., 2025d ) . Furthermore, advanced testbeds treat the feedback mechanism itself as a mutable artifact, with environments like SWT-bench and TDD-bench requiring agents to autonomously generate and refine the very unit tests driving their own improvement loops ( Mündler et al., 2024 ; Ahmed et al., 2024 ) .

##### Web Navigation and Automation.

Web environments are dynamic and sensitive to superficial layout cues, so evaluation should carefully separate genuine generalization from mere memorization. Protocols benefit from controlled websites or environment randomization, and they track whether extraction errors compound when written into memory. Benchmarks include controlled execution-based suites such as WebArena and VisualWebArena ( Zhou et al., 2024 ; Koh et al., 2024 ) , transfer-oriented suites on real websites such as Mind2Web-style benchmarks ( Deng et al., 2023 ; Pan et al., 2024 ) , and enterprise settings that evaluate policy compliance under continuous improvement ( Levy et al., 2026 ) .

##### Gaming and Strategic Reasoning.

Games expose learning curves naturally, but evaluation should guard against overfitting to a fixed opponent pool. Protocols employ diverse opponents, cross-play evaluation, and held-out strategies ( 𝒟 eval \mathcal{D}_{\text{eval}} ). Representative suites include dynamic instance generation for interactive agents ( Chalamalasetti et al., 2023 ; Beyer et al., 2024 ) , head-to-head evaluation for robustness ( Costarelli et al., 2024 ) , and negotiation environments with quantitative outcomes ( Abdelnabi et al., 2024 ; Duan et al., 2024 ) .

##### Scientific Discovery.

In scientific discovery, improvement can be tied to verifiable artifacts such as executable analyses, reproducible results, and traceable provenance. Evaluation should prioritize execution-based checks ( Φ metric \Phi_{\text{metric}} ) and report wall-clock time and operational cost alongside artifact quality. Benchmarks include reproducibility-focused execution tasks ( Siegel et al., 2024 ) , controlled research-agent suites ( Bragg et al., 2026 ) , and protocols that grade iterative progress and replication quality ( Starace et al., 2025 ) . Simulated discovery environments provide controlled testbeds for long-horizon evaluation ( Jansen et al., 2024 ; Chen et al., 2025b ) .

##### Embodied AI.

Embodied evaluation handles severe distribution shifts and physical safety constraints that limit free exploration. Protocols report success rates under controlled environmental shifts, and crucially track safety constraint violations and recovery behavior across iterations. Representative benchmarks include safety-oriented suites ( Yin et al., 2025a ) , transfer-oriented manipulation benchmarks ( Gu et al., 2023 ) , and multi-modal embodied benchmarks that stress grounding and generalization ( Yang et al., 2025b ) .

##### General Computer Control.

Desktop agents combine heterogeneous applications, long contexts, and fragile user interface feedback. Evaluation should report reliability, error-recovery capabilities, latency, and operational cost under standardized virtual machines. Benchmarks include operating system level virtual machine suites with programmatic evaluators ( Xie et al., 2024 ; Bonatti et al., 2025 ) , controllable application simulators with state-based scoring ( Trivedi et al., 2024 ) , and tool-use suites that test tool invocation, selection, grounding, and safety failure modes ( Patil et al., 2025 ; Huang et al., 2024b ; Ruan et al., 2024a ) .

## 9 Discussion

Self-improving agents are closed-loop dynamical systems ( Xie et al., 2024 ) . According to our formulation, an agent 𝒜 t = ( θ t , Σ t ) \mathcal{A}_{t}=(\theta_{t},\Sigma_{t}) progresses by executing a signal-generation procedure and applying a stable update rule to induce new policy. Consequently, the goal of the study is no longer the static agent but the mechanism driving its evolution ( Yampolskiy, 2015 ; Robeyns et al., 2025 ; Pan et al., 2025 ) . Building on the rigorous evaluation protocols established in Section 8 , we examine the architecture of such systems and outline key directions for future research.

### 9.1 Implications for System Design

##### From Fast Exploration to Slow Consolidation.

Modifying the operational scaffold ( Σ \Sigma ) drives a fast adaptation loop. Prompt edits, memory writes, and tool adjustments have minimal computational overhead and are reversible. In contrast, parameter updates ( θ \theta ) are much slower. While weight modifications excel at transferring capabilities across distinct domains, they inherently obscure credit assignment: a bad prompt is easily reverted, but a regression absorbed into model parameters is notoriously difficult to trace.

This structural asymmetry necessitates specific deployment strategies. When environmental feedback is noisy, it’s better to confine updates within the scaffold and validate them through rigorous execution tests. Parametric consolidation (through distillation or fine-tuning) can be deferred until the new behavior has proven stable ( Qiu et al., 2025a ; Lyu et al., 2026 ) . However, parametric consolidation is a lossy compression. Distilling complex trajectories into neural weights inherently biases the model toward average-case execution. This process often discards the rare but crucial error-recovery strategies discovered during exploration. As a result, any update to θ \theta invalidates prior safety bounds, necessitating renewed adversarial testing to verify that edge-case resilience was preserved ( Dou et al., 2024 ; Wu et al., 2024a ) .

##### The Critic as Governed Infrastructure.