MemEvolve: Meta-Evolution of Agent Memory Systems — part 1/3

Cleaned-text code points [0, 15974). Bibliography omitted here; source.md includes it.

\checkdata [ Code] https://github.com/bingreeky/MemEvolve

# MemEvolve: Meta-Evolution of Agent Memory Systems

###### Abstract

Self-evolving memory systems are unprecedentedly reshaping the evolutionary paradigm of large language model (LLM)-based agents. Prior work has predominantly relied on manually engineered memory architectures to store trajectories, distill experience, and synthesize reusable tools, enabling agents to evolve on the fly within environment interactions. However, this paradigm is fundamentally constrained by the staticity of the memory system itself: while memory facilitates agent-level evolving, the underlying memory architecture cannot be meta-adapted to diverse task contexts. To address this gap, we propose MemEvolve , a meta-evolutionary framework that jointly evolves agents’ experiential knowledge and their memory architecture, allowing agent systems not only to accumulate experience but also to progressively refine how they learn from it. To ground MemEvolve in prior research and foster openness in future self-evolving systems, we introduce EvolveLab , a unified self-evolving memory codebase that distills twelve representative memory systems into a modular design space ( encode , store , retrieve , manage ), providing both a standardized implementation substrate and a fair experimental arena. Extensive evaluations on four challenging agentic benchmarks demonstrate that MemEvolve achieves (I) substantial performance gains, improving frameworks such as SmolAgent and Flash-Searcher by up to 17.06 % 17.06\% ; and (II) strong cross-task and cross-LLM generalization, designing memory architectures that transfer effectively across diverse benchmarks and backbone models.

## 1 Introduction

Language agents and agent systems, empowered by increasingly capable foundation models ( Team et al., 2025a, ; Team et al., 2025b, ) and sophisticated scaffolding ( Wang et al., 2024a, ; LangChain,, 2023 ) , have advanced rapidly, demonstrating unprecedented performance across complex tasks such as deep research ( Chen et al.,, 2025 ) , scientific discovery ( Bai et al.,, 2025 ; Wei et al., 2025b, ) , and industrial report generation ( Zhang et al., 2025g, ) . A key driving force behind this success is the agent memory system ( Zhang et al., 2024b, ; Hu et al., 2025c, ) , which persistently captures interactions between the agent and environment, distilling them into diverse forms of knowledge and skills, and thereby enabling large language model (LLM)-based agents to evolve continuously in task solving and world exploration ( Wu et al., 2025c, ) .

Naturally, the choice of memory paradigm plays a decisive role in shaping an agent’s capacity for on-the-fly self-evolution. Initial designs centered on raw trajectory storage and few-shot prompting ( Zhong et al.,, 2024 ; Wen et al.,, 2024 ) , which were later superseded by more abstracted textual artifacts such as tips, shortcuts, and reasoning templates ( Ouyang et al.,, 2025 ; Zhang et al., 2025b, ; Ye et al.,, 2025 ; Tang et al.,, 2025 ) . Recent advances have also explored structured tool interfaces ( e.g. , APIs ( Zheng et al.,, 2025 ) , MCPs ( Qiu et al., 2025b, ; Qiu et al., 2025a, ; Zhang et al., 2025h, ) ) and code-level repositories ( Zhang et al., 2025e, ; Wang et al., 2025a, ) as memory carriers. Amid this growing diversity, an inquisitive practitioner might ask: What kind of memory architecture most effectively drives agent self-improving?

We posit that no universally optimal memory architecture exists. For instance, a memory system that distill reusable APIs from past trajectories may excel in tasks such as web browsing, yet offer limited utility for mathematical and scientific reasoning. Conversely, memories predicated on self-critique, while powerful in reasoning-intensive domains ( Cai et al.,, 2025 ) , show diminished efficacy in coding and tool-use scenarios, as empirically discussed in ( Zhang et al., 2025d, ) . We contend that these trade-offs arise from the static nature of current memory systems. Researchers typically design a fixed memory pipeline ( i.e. , memory ingestion/abstraction/retrieval ( Zhang et al., 2025i, ) ) and embed it within an agent, assuming it will sustain long-term evolution through mere exposure to new experiences. Yet this overlooks a crucial reality: distinct tasks are coupled with distinct memory affordances. A memory system that cannot adapt itself to the task at hand is fundamentally misaligned with the very premise of open-ended agent evolution.

To elucidate this dilemma, consider the analogy of human learning. Both high- and low-performing students inevitably make mistakes, yet their distinction lies in the meta-cognitive strategies they employ to learn from these errors. An underperforming student might resort to rote memorization, superficially recording an error without genuine comprehension ( Zhong et al.,, 2024 ; Orhan,, 2023 ) . In contrast, a more skillful student engages in higher-order learning: they not only record errors but also distill transferable insights through reflection ( Shinn et al.,, 2023 ; Zhao et al.,, 2024 ) or derive reusable schemas ( Zheng et al.,, 2025 ; Qiu et al., 2025b, ) ). Current memory systems effectively model a skillful learner. Herein lies the critical gap: the most effective human learners are not merely skillful, but adaptive . They dynamically alter their learning strategies based on the subjects, for instance, prioritizing memorization for literary analysis while abstracting solution templates for mathematics. It is precisely this transition, from a skillful to an adaptive learner (as shown in Figure 2 ), that we argue agent memory systems must undergo. To put it more formally:

To address the challenge, we introduce MemEvolve , a framework that facilitates the dual evolution of an agent’s experience and its memory architecture. Conceptually, MemEvolve operates as a bilevel optimization process: the inner loop performs a first-order evolution , where the agent, guided by a fixed memory system, adapts to a continuous stream of new tasks by populating its experience base. The outer loop drives a second-order evolution , meta-learning a more effective memory architecture to accelerate future learning. This allows the agent not only to evolve, but to evolve more efficiently and intelligently over time.

However, the vast and heterogeneous design space of memory systems ( e.g. , knowledge graphs, skill libraries, vector databases) presents a significant challenge to controllable optimization. To render this optimization tractable, we introduce a modular design, decomposing any memory architecture into four key components: ♣ Encode (perceiving and formatting experiences), ♠ Store (committing information), ♥ Retrieve (context-aware recall), and ♠ Manage (consolidation and forgetting). MemEvolve evolves the programmatic implementations of these modules in a model-driven fashion, using feedback from the agent’s performance in the inner loop. This process establishes a virtuous cycle: an improved memory architecture from the outer loop enhances the agent’s learning efficiency. In turn, a more capable agent generates higher-quality trajectories, providing a more precise fitness signal for the outer loop to drive the next round of architectural evolution.

To ground our framework within the diverse landscape of existing self-improving agent memories, we systematically re-implement twelve representative architectures in a unified modular design space, including ExpeL ( Zhao et al.,, 2024 ) , Agent Workflow Memory ( Wang et al., 2024b, ) , and Dynamic Cheatsheet ( Suzgun et al.,, 2025 ) . The resulting framework, denoted as EvolveLab , serves both as an empirical foundation for MemEvolve ’s evolutionary process and as a standardized codebase to facilitate future research on self-evolving agents. Our contributions are as follows:

❶ Unified Codebase: We introduce EvolveLab , a modular design space for self-improving agent memory systems encompassing four key components ( encoding , storage , retrieval , and management ), providing unified implementations and benchmark support for a wide range of prevailing agent memory systems.

❷ Meta-Evolution Framework: We propose MemEvolve , a meta-evolutionary framework that jointly evolves both agents’ experiential knowledge and their underlying memory architecture, in which agent systems not only accumulate experience but also progressively refine their mechanism for learning from it.

❸ Experimental Evaluation: Extensive experiments on four challenging agentic benchmarks demonstrate that MemEvolve delivers (I) substantial performance gains , improving frameworks such as SmolAgent and Flash-Searcher by up to 17.06 % 17.06\% ; and (II) cross-domain, cross-framework and cross-LLM generalization , where memory systems evolved on TaskCraft yield 2.0 − 9.09 % 2.0-9.09\% gains with unseen benchmarks and backbone models.

## 2 Related Work

LLM Agent Systems. The past two years have witnessed rapid advances in LLM-based agent systems across multiple dimensions ( Tran et al.,, 2025 ; Fang et al., 2025a, ) . In terms of system complexity , development has progressed from early single-agent setups with manually defined workflows and limited tool configurations ( Wu et al.,, 2023 ; Significant-Gravitas,, 2023 ) to sophisticated multi-agent architectures featuring diverse MCP integrations and automated orchestration ( Zhang et al., 2024a, ; Zhang et al., 2025a, ; Wang et al., 2025b, ; Zhang et al., 2025c, ) . From the perspective of task domains , capabilities have expanded from relatively constrained areas such as coding and mathematical reasoning ( Hong et al.,, 2024 ; Yin et al.,, 2023 ) to more challenging domains, including deep research and scientific discovery ( Du et al.,, 2025 ; Ghareeb et al.,, 2025 ) . Today, numerous open-source multi-agent systems demonstrate competitive performance on demanding benchmarks such as GAIA ( Mialon et al.,, 2023 ) , HLE ( Phan et al.,, 2025 ) , BrowseComp ( Wei et al., 2025a, ) , and xBench ( Chen et al.,, 2025 ) , including CAMEL’s OWL ( Hu et al., 2025a, ) , Tencent’s CK-Pro ( Fang et al., 2025c, ) , Skywork’s AgentOrchestra ( Zhang et al., 2025f, ) , and ByteDance’s AIME ( Shi et al., 2025b, ) , among others.

Agent Memory Architectures. Agent memory systems can be broadly divided by objective into personalized memory and self-improving memory ( Zhang et al., 2024b, ; Hu et al., 2025c, ) . The former enables agent chatbots to dynamically capture user-specific information and preferences, while the latter focuses on distilling knowledge and skills from continual interactions with the environment to enhance performance, a focus adopted in this work. Self-improving memories are primarily differentiated by their storage modality. Early systems stored raw agent trajectories as few-shot examples ( Wang et al.,, 2023 ; Zhong et al.,, 2024 ; Packer et al.,, 2023 ) ; subsequent designs abstracted these experiences into higher-level lessons, insights ( Yang et al.,, 2025 ; Sun and Zeng,, 2025 ; Wu et al., 2025b, ) , procedural tips ( Wang et al., 2025c, ; Zheng et al.,, 2025 ; Fang et al., 2025b, ) , and more recently, reusable tools and structured repositories ( Zhao et al.,, 2025 ; Qiu et al., 2025a, ; Qiu et al., 2025b, ; Zhang et al., 2025e, ) . Despite their differences in representation, there approaches share the same ambition, i.e. , to enable agents to learn, adapt, and improve in a human-esque manner.

## 3 EvolveLab: A Unified Codebase for Self-Evolving Memory

In this section, we first formalize the LLM-based agentic system and its associated memory architecture, then present the modular design space of EvolveLab , which comprehensively captures the characteristics of existing self-evolving agent memories, and finally introduce the unified codebase EvolveLab .

### 3.1 Preliminary

We formalize an LLM-based agentic system as ℳ = ⟨ ℐ , 𝒮 , 𝒜 , Ψ , Ω ⟩ \mathcal{M}=\langle\mathcal{I},\mathcal{S},\mathcal{A},\Psi,\Omega\rangle , where ℐ \mathcal{I} indexes the { 1 , ⋯ , N } \{1,\cdots,N\} agents, 𝒮 \mathcal{S} denotes the shared state space, 𝒜 = ⋃ i ∈ ℐ 𝒜 i \mathcal{A}=\bigcup_{i\in\mathcal{I}}\mathcal{A}_{i} represents the joint action space, and Ψ ⁡ ( s t + 1 ∣ s t , a t , μ ⁡ ( t ) ) \Psi(s_{t+1}\mid s_{t},a_{t},\mu(t)) describes the environment dynamics with μ ⁡ ( t ) ∈ ℐ \mu(t)\in\mathcal{I} indicating the active agent at time step t t . The system leverages a memory module Ω \Omega , which maintains a continuously evolving memory state M t M_{t} . At each step, the active agent observes the current state s t s_{t} , considers a task-specific query 𝒬 \mathcal{Q} , and interacts with Ω \Omega to retrieve contextually relevant memory c t c_{t} , conditioned on its interaction history ℋ t \mathcal{H}_{t} . The agent μ t \mu_{t} ’s policy π μ t \pi_{\mu_{t}} then delivers an action: a t = π μ ⁡ ( t ) ​ ( s t , ℋ t , 𝒬 , c t ) , c t ∼ Ω ⁡ ( M t , s t , ℋ t , 𝒬 ) . a_{t}=\pi_{\mu(t)}(s_{t},\mathcal{H}_{t},\mathcal{Q},c_{t}),\;c_{t}\sim\Omega(M_{t},s_{t},\mathcal{H}_{t},\mathcal{Q}).

Following task execution, a trajectory τ = ( s 0 , a 0 , … , s T ) \tau=(s_{0},a_{0},\dots,s_{T}) is recorded, with an overall performance evaluated via a terminal reward R ⁡ ( τ ) R(\tau) . The memory system assimilates new experience units ϵ \epsilon , which can vary in granularity (from individual state-action transitions to aggregated segments or complete trajectories), and updates the memory state as M t + 1 = Ω ⁡ ( M t , ϵ ) , M_{t+1}=\Omega(M_{t},\epsilon), where Ω \Omega abstracts the memory’s mechanisms for integrating and organizing new experiences or knowledge.

### 3.2 Modular Design Space of Memory Systems

The heterogeneous and rapidly evolving landscape of self-improving agent memories presents challenges for systematic analysis and controlled experimentation. To address this, we propose a modular design space that decomposes any memory system Ω \Omega into four functionally distinct yet interdependent components: Ω = ( ℰ , 𝒰 , ℛ , 𝒢 ) \Omega=(\mathcal{E},\mathcal{U},\mathcal{R},\mathcal{G}) , representing encode , store , retrieve , and manage operations, respectively.

• Encode ( ℰ \mathcal{E} ) : Transforms raw experiences, such as trajectory segments τ t = ( s t , a t , s t + 1 ) \tau_{t}=(s_{t},a_{t},s_{t+1}) , tool outputs, or self-critiques, into structured representations e t = ℰ ⁡ ( ϵ t ) e_{t}=\mathcal{E}(\epsilon_{t}) . Encoding may be as simple as compressing raw traces ( Zheng et al.,, 2023 ) or as sophisticated as extracting generalizable lessons ( Zheng et al.,, 2025 ) .

• Store ( 𝒰 \mathcal{U} ) : Integrates encoded experiences into the persistent memory M t M_{t} , yielding M t + 1 = 𝒰 ⁡ ( M t , e t ) M_{t+1}=\mathcal{U}(M_{t},e_{t}) . Storage can be vector databases ( Zhao et al.,, 2024 ) , knowledge graphs ( Zhang et al., 2025b, ; Rasmussen et al.,, 2025 ) , or others.

• Retrieve ( ℛ \mathcal{R} ) : Provides task-relevant memory content, formalized as c t = ℛ ⁡ ( M t , s t , 𝒬 ) c_{t}=\mathcal{R}(M_{t},s_{t},\mathcal{Q}) , which informs the agent’s policy decision a t a_{t} . Retrieved content may include reusable tools ( Zhang et al., 2025f, ) , planning experience ( Tang et al.,, 2025 ) , or distilled procedural knowledge ( Wu et al., 2025b, ; Yang et al.,, 2025 ; Fang et al., 2025b, ) .

• Manage ( 𝒢 \mathcal{G} ) : Performs offline and asynchronous operations such as consolidation, abstraction, or selective forgetting to maintain long-term memory quality and efficiency, denoted as M t ′ = 𝒢 ⁡ ( M t ) M^{\prime}_{t}=\mathcal{G}(M_{t}) .

This modular abstraction allows us to represent each memory system as a specific combination of programmatic implementations for ( ℰ , 𝒰 , ℛ , 𝒢 ) (\mathcal{E},\mathcal{U},\mathcal{R},\mathcal{G}) , forming a “genotype” that facilitates the meta-evolutionary process of MemEvolve .

### 3.3 EvolveLab Codebase