Self-Improvements in Modern Agentic Systems: A Survey

arxiv:2607.13104

Original source · Paper JSON

Agents, open-endedness, AGI · 2026-07-14

Cites (14) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4
5. Part 5
6. Part 6
7. Part 7
8. Part 8
9. Part 9
10. Part 10
11. Part 11
12. Part 12
13. Part 13
14. Part 14

Contents

• Self-Improvements in Modern Agentic Systems: A Survey
• Abstract
• 1 Introduction
• 2 Historical Context and Theoretical Foundations
• 2.1 Foundational Concepts (1790s-1960s)
• 2.2 Symbolism and Heuristic Self-Modification (1960s–1980s)
• 2.3 Connectionism and the Emergence of Meta-Learning (1980s–2000s)
• 2.4 Formal and Architecture-Level Self-Improvement (2000s-2020s)
• 2.5 Scalable Foundation Models and Agentic Systems (2020s–Present)
• 3 Definitions
• 3.1 Formulation of Agentic Systems
• 3.2 Formal Definition of Self-Improvement
• Self-improvement as a self-induced operator.
• Two modes of self-reference in SI-FMA.
• Foundation model improvement.
• Scaffolding improvement.
• Skill as a reusable update.
• 3.3 Connections to Related Learning Paradigms
• 4 A Taxonomy of Existing Approaches
• 4.1 Foundation Model Improvement
• Classification by signal form.
• 4.2 Scaffolding Improvement
• Classification by scaffold component.
• 5 Foundation Model Improvement
• 5.1 Intrinsic Generative Demonstrations
• Generation strategies.
• Data formats.
• Challenges and safeguards.
• 5.2 Intrinsic Evaluative Feedback
• Rubric feedback.
• Consistency feedback.
• Corrective feedback.
• Trade-offs and safeguards.
• 5.3 Extrinsic Exploratory Experience
• 5.3.1 Interaction with Grounded Task Environments.
• 5.3.2 Interaction with Simulated Proxy Environments
• Challenges.
• 6 Scaffolding Improvement
• 6.1 Prompt
• 6.1.1 Scalar-Feedback Optimization
• 6.1.2 Qualitative-Feedback Refinement
• 6.1.3 Population-Based Evolution
• 6.1.4 Textual Gradient Optimization
• 6.2 Memory
• 6.2.1 Memory Object
• Trade-offs.
• 6.2.2 Memory Structure
• 6.2.3 Memory Processing
• Putting it together: a signal-driven memory loop.
• 6.3 Tool
• 6.3.1 Dynamic Tool Routing
• Retrieval- and graph-based routing.
• Policy-learning routing.
• Proactive and interactive routing.
• 6.3.2 Iterative Tool Refinement
• 6.3.3 Autonomous Tool Creation
• 6.4 Full Scaffolding
• 7 Applications
• 7.1 Software Engineering
• Evaluation substrate vs. self-improvement.
• Scaffolding improvement via self-modifying software agents.
• FM improvement from executable feedback.
• Synthesis and open challenges.
• 7.2 Web Navigation and Automation
• Evaluation and data substrate.
• Scaffolding improvement with memory and grounding.
• FM improvement from interaction feedback.
• Synthesis and open challenges.
• 7.3 Games and Strategic Reasoning
• FM improvement through self-play and outcome feedback.
• Scaffolding improvement through curricula and reusable skills.
• Synthesis and open challenges.
• 7.4 Scientific Discovery
• Evaluation substrate and feedback structure.
• Scaffolding improvement through tool expansion and workflow evolution.
• Model improvement through closed-loop experimentation.
• Synthesis and open challenges.
• 7.5 Embodied AI and Robotics
• Evaluation substrate and feedback structure.
• FM improvement through autonomous practice and data flywheels.
• Scaffolding improvement through curricula, memory, and safety logic.
• Synthesis and open challenges.
• 7.6 General Computer Control
• Evaluation substrate.
• Scaffolding improvement via experience and procedural reuse.
• FM improvement through iterative training and verifier construction.
• Synthesis and open challenges.
• 8 Evaluation
• 8.1 Measuring Improvement
• Formalizing the Evaluation Objective.
• 8.1.1 Metric-Based Measurement
• 8.1.2 Judge-Based Measurement
• Specify the judge and the judging budget.
• Prevent over-optimization to the judge and report reliability signals.
• 8.2 Benchmarking Improvement
• 8.2.1 Mechanism Benchmarks
• FM -Level Evaluation.
• Scaffold -Level Evaluation.
• Attribution Across Mechanisms.
• 8.2.2 Domain Benchmarks
• Software Engineering.
• Web Navigation and Automation.
• Gaming and Strategic Reasoning.
• Scientific Discovery.
• Embodied AI.
• General Computer Control.
• 9 Discussion
• 9.1 Implications for System Design
• From Fast Exploration to Slow Consolidation.
• The Critic as Governed Infrastructure.
• Safety through Layered Gating.
• 9.2 Future Directions
• 10 Conclusion