Self-Improvements in Modern Agentic Systems: A Survey — part 1/14

Cleaned-text code points [0, 15376). Bibliography omitted here; source.md includes it.

# Self-Improvements in Modern Agentic Systems: A Survey

###### Abstract

Self-improving autonomous agents are moving from research prototypes to deployed systems. The primary goal is controllable evolution, or adaptation, from experience with minimal or even no human input. This survey frames modern self-improving agents as adaptive systems that convert experience into accumulated capability gains. We offer a system-level framework that represents a modern agent as a configuration coupling a foundation model with an operational scaffold of prompts, memory, tools, and control logic. Within this framework, self-improvement is formalized as a self-induced update operator that obtains and commits updates to model parameters or scaffold components. We organize prior work by update target and by the signals that drive change, then review applications and discuss evaluation, before closing with open problems and future directions. For convenience, we track technical updates on this GitHub page .

Self-Improving-Agents Project Page

## 1 Introduction

The development of artificial intelligence (AI) technology has driven a paradigm shift in agentic systems ( Shoham, 1993 ; Maes, 1994 ; Wooldridge and Jennings, 1995 ; Yao et al., 2022 ; Wang et al., 2024a ) , from earlier narrow systems built around task-specific models or hand-engineered modules to modern agentic systems powered by foundation models (FMs), including large language models (LLMs) and vision-language models (VLMs), where natural language serves as a shared interface for representation, reasoning, and control. Progress in foundation models has produced a qualitative shift in generalization, yielding striking successes across a wide range of domains, most notably code generation ( Chen et al., 2021 ) , language understanding ( Hendrycks et al., 2020 ) , and mathematical and formal reasoning ( Wei et al., 2022 ) . These advances have brought a long-standing question to the foreground: the prospect of AI systems that improve themselves. Fundamentally, self-improvement is an inherently self-referential process. It defines a system’s capacity to autonomously inspect, evaluate, and deliberately modify its own underlying optimization mechanisms and operational logic. Good articulated possible consequences of machine self-improvement ( Good, 1966 ) , describing the possibility of an “Intelligence Explosion” once machines acquire the capacity to design more capable successors. Early work on concrete self-improvement algorithms dates back to Schmidhuber’s self-referential learning framework ( Schmidhuber, 1987 ) , which introduced mechanisms in which a system generates and evaluates modified descendant versions of itself. Establishing the theoretical ceiling of this pursuit, the Gödel Machine ( Schmidhuber, 2003b ) introduced a fully self-referential algorithm designed to rewrite its own code whenever it can mathematically prove an expected-utility improvement. While a persistent lineage of research successfully demonstrated neural networks (NNs) learning to program other networks via fast weights ( Schmidhuber, 1992 ) , advancing to self-referential systems capable of modifying themselves ( Schmidhuber, 1993 ; Irie et al., 2022 ) , and acting as meta-learning systems to discover their own learning algorithms ( Hochreiter et al., 2001 ; Schmidhuber, 2004 ; Kirsch and Schmidhuber, 2021 ; Kirsch and Schmidhuber, 2022 ; Irie et al., 2025 ) , parallel efforts introduced incremental self-improvement, establishing the ability to enforce long-term reward accelerations through the success-story algorithm for undoing policy self-modifications through backtracking ( Schmidhuber, 1994 ; Schmidhuber et al., 1996 ; Schmidhuber et al., 1997 ) . However, scaling these visionary mechanisms into open-ended agents was historically constrained. Traditionally, systems were forced to search through vast, low-level spaces of assembly-like code or raw synaptic weights. Today, modern FMs alleviate this historical bottleneck by introducing natural language as a unified, highly capable semantic medium for reasoning, policy execution, and self-modification. By drastically reducing the search space for viable modifications, this language-native paradigm has drawn intense recent attention, with growing evidence demonstrating that FM-based self-improving agents can yield substantial empirical gains ( Yuan et al., 2024b ; Acikgoz et al., 2025 ; Qi et al., 2025 ; Zhang et al., 2026c ) .

To function autonomously in concrete environments, the FM serving as the cognitive core is typically enveloped by an operational scaffold, a structured framework comprising instruction schemes ( Yuksekgonul et al., 2024 ; Guo et al., 2024 ) , memory systems ( Chhikara et al., 2025 ; Zhang et al., 2026d ) , tool interfaces ( Qiu et al., 2025c ; Wölflein et al., 2025 ) , and control logic ( Hong et al., 2023 ; Zhuge et al., 2024a ; Xiong et al., 2025 ) . Recently, such an operational scaffold is often also referred to as an agent harness ( Rajasekaran, 2026 ; Trivedy, 2026 ; Zhang et al., 2026b ; Zhong and Zhu, 2026 ) , while this survey uses the term scaffold to emphasize the modifiable structures surrounding the foundation model. Operationally, this scaffold acts as a controller that constructs context, selects actions, and enforces constraints. This modern architecture was conceptually foreshadowed by the "learning to think" framework ( Schmidhuber, 2015 ) , where a general-purpose controller learned to dynamically query, or "prompt," a predictive world model to generate reasoning sequences akin to a modern "chain of thought" ( Wei et al., 2022 ) . Building upon this core-and-scaffold architecture, the integration of these classical control principles has crystallized into an emergent paradigm, which we refer to as FM-based self-improving agents. Because modern agentic systems comprise both the aforementioned neural core and scaffold, we organize this survey around a unified taxonomy with two primary pathways, as illustrated in Fig. 1 and summarized chronologically in Fig. 2 . The first pathway, which we call foundation model improvement , aims to achieve a slower but more stable form of long-term consolidation by updating the underlying model, thereby amortizing capability gains across varied tasks ( Huang et al., 2023 ; Zhao et al., 2024b ; Wang et al., 2025c ; Xiao et al., 2025 ; Bougie et al., 2026 ) . The second pathway, which we call scaffold improvement , is typically faster and more easily reversible; it improves the agent by updating structural components, including prompts, memory, tool interfaces, and end-to-end control logic, to reshape the agent’s effective observation and action semantics ( Fernando et al., 2024 ; Chhikara et al., 2025 ; Wölflein et al., 2025 ; Zhang et al., 2026c ; Borthwick and Ash, 2026 ) . Although these pathways differ in their targets of modification, both are fundamentally driven by learning signals extracted during interaction. Accordingly, our taxonomy further refines these methods by separating what is updated from where the improvement signals originate, providing a common language for comparing disparate methods on a consistent footing. Fig. 3 provides a unified taxonomy of this survey.

✓ \checkmark primary ∘ \circ secondary – not focus

Despite the rapid proliferation of such empirical frameworks, the broader research landscape remains highly fragmented in both terminology and scope. Closely related ideas are often described under different labels (e.g., self-correction, meta-prompting, or self-play), obscuring underlying mechanistic similarities. Recent surveys have provided valuable perspectives by organizing self-evolving agents along dimensions such as what to evolve, when to evolve, and how to evolve ( Gao et al., 2026 ; Fang et al., 2025a ) , or by focusing specifically on the autonomous learning capabilities of static LLMs ( Tao et al., 2024 ) . However, existing reviews often treat foundation model fine-tuning and agent scaffolding as isolated topics, lacking a unified formal perspective. Furthermore, few trace the conceptual roots of self-improvement back to classical AI. To bridge this gap, our survey provides a rigorous and systematic positioning of these modern agentic systems. We offer a comprehensive taxonomy under a unified formalization, clarifying their historical evolution and providing a clear, forward-looking roadmap for self-improvement. Concretely, our main contributions are as follows: • Historical Context and Evolution. We trace the evolutionary roots of self-improving systems from classical AI to modern FM-based agents, establishing a clear trajectory for how foundational learning mechanisms have adapted to the agentic era.

• A Unified Formalization and Systematic Taxonomy. Under our proposed formulation, we systematically categorize mechanisms into two distinct pathways: foundation model improvement and scaffold improvement (covering prompt optimization, memory evolution, tool governance, and full-scaffold redesign).

• Empirical Landscape, Evaluation Paradigms, and Frontiers. We integrate the classification system with practical applications (such as software engineering, web navigation, and scientific discovery) and critically analyze current evaluation protocols. By comparing mechanism-level benchmarking and end-to-end evaluation, we highlight the security factors and challenges that need to be considered in achieving reliable, continuous self-improvement.

The remainder of this survey is organized as follows. Section 2 traces the historical context and theoretical foundations of self-improving systems. Section 3 introduces the systems view of foundation-model-based agents and formalizes the problem setting. Section 4 presents our unified taxonomy, outlining the two primary pathways for improvement. These pathways are subsequently surveyed in depth: Section 5 covers updates to the underlying foundation model, while Section 6 focuses on structural modifications to the surrounding scaffold. Section 7 reviews practical instantiations across representative domains. Section 8 discusses protocols and benchmarks for evaluating these systems. Finally, Section 9 synthesizes open problems and safety considerations, followed by concluding remarks in Section 10 .

## 2 Historical Context and Theoretical Foundations

Self-improvement is not unique to modern foundation models; it is a foundational objective of artificial intelligence. Whereas standard machine learning optimizes parameters within a fixed architecture, true self-improvement demands that a system explicitly inspect and rewrite its own operational logic, heuristics, or learning algorithms. As illustrated in Figure 4 , the mechanisms have been successfully instantiated across several historical paradigms. By tracing this intellectual lineage, we clarify how earlier systems achieved self-improvement within bounded domains, and how modern foundation models now provide a highly expressive, general-purpose substrate to scale these established principles into open-ended and real-world environments.

### 2.1 Foundational Concepts (1790s-1960s)

The mathematical roots of error-driven adaptation extend back to early optimization frameworks. The method of least squares ( Legendre, 1805 ; Gauss, 1809 ) 1 1 1 The first famous example of pattern recognition through an NN dates back over 200 years: the rediscovery of the dwarf planet Ceres in 1801 through Gauss, who collected noisy data points from previous astronomical observations, then used them to adjust the parameters of a predictor, which essentially learned to generalize from the training data to correctly predict the new location of Ceres ( Schmidhuber, 2022 ) . demonstrated how parameters of what’s now known as “linear neural networks” can be systematically adjusted to minimize errors. As highlighted in historical retrospectives, this established a mechanism that is still heavily used today and serves as the very foundation of all neural networks, including deeper ones ( Schmidhuber, 2022 ) .

Throughout the mid-20th century, these concepts of adaptation continued to evolve, acting as conceptual precursors. Cybernetics popularized the feedback view of adaptive behavior in closed-loop systems ( Wiener, 1948 ) , while Ashby’s work on homeostasis emphasized internal state adjustment to maintain viability under perturbations ( Ashby, 1952 ) . Alan Turing’s “child machine” proposed a model whose internal configuration could be shaped through training and external feedback, such as reward and punishment ( Turing, 1948 ; Turing, 1950 ) . Early systems, from Rosenblatt’s perceptron ( Rosenblatt, 1958 ) to Samuel’s checkers program ( Samuel, 1959 ) , provided concrete demonstrations of how performance feedback could change future behavior. However, while these mid-century milestones demonstrated that behavior could optimize over time, their underlying architectures and learning algorithms themselves remained strictly fixed. Consequently, they are best understood as advanced learning systems, rather than fully self-referential, self-improving systems.

The formal basis for transcending fixed learning rules and enabling true self-reference followed a distinct trajectory. In the early 1930s, Kurt Gödel founded modern theoretical computer science and identified fundamental limits of any type of computation-based AI ( Gödel, 1931 ) . He introduced a universal coding language based on integers, using it to represent both data and programs in axiomatic form. By famously constructing formal statements that talk about the computation of other formal statements—especially self-referential statements—Gödel provided the theoretical blueprint for programs capable of manipulating their own code ( Schmidhuber, 2022 ) . In the 1960s, Good ( Good, 1966 ) introduced the visionary prospect of an “Intelligence Explosion,” hypothesizing that machines might one day acquire the capacity to autonomously design more capable successors. However, while this narrative highlighted self-improvement as an ultimate pursuit in AI, it remained a conceptual idea without a formal mathematical or algorithmic framework. Translating this speculative concept into actual deployable mechanisms would require the subsequent development of explicit structural and algorithmic self-modification frameworks.

### 2.2 Symbolism and Heuristic Self-Modification (1960s–1980s)

As symbolic AI rose to prominence, self-improvement was reinterpreted as the ability to manipulate explicit representations of knowledge and strategy. A key conceptual precursor was von Neumann’s theory of self-reproducing automata. Originating in the late 1940s and later published in 1966, it distinguished a constructive mechanism from a symbolic description that can be copied and varied ( Von Neumann et al., 1966 ) . Myhill (1964) abstracted self-reproduction into a rigorous formal framework, emphasizing how systems can use symbolic descriptions to generate copies or variants of themselves. Together, these concepts clarified how a system can generate successive versions of increasing complexity by editing the descriptions it interprets. A similar approach in symbolic artificial intelligence is to treat problem-solving heuristics as first-class objects that can be examined, modified, and recombined.