Self-Improvements in Modern Agentic Systems: A Survey — part 8/14

Cleaned-text code points [107255, 122524). Bibliography omitted here; source.md includes it.

The memory system serves as the core cognitive scaffolding for long-horizon agentic behavior. Traditional memory architectures often rely on raw content logging alongside fixed schemas, resulting in rigid, static organizations. Such append-only designs quickly succumb to context-window constraints and retrieval degradation, rendering them ill-suited for dynamically changing environments ( Modarressi et al., 2024 ; He et al., 2024a ; Packer et al., 2024 ) . In contrast, self-improving agents treat memory not merely as a passive storage mechanism, but as an actively evolving scaffold. By continuously assessing the value, relevance, and strength of stored information, these agents autonomously reconstruct and expand their memory representations based on the flow of experience ( Du et al., 2025 ; Xu et al., 2025b ) . This shift from static storage to dynamic self-organization enhances the agent’s generality, establishing a foundation for open-ended autonomy ( Xu et al., 2025f ; Sang et al., 2025 ; Li et al., 2025d ) .

To systematically analyze this paradigm, we decompose memory-based improvement into three core dimensions: Memory Objects (the units of stored information), Memory Structure (the topological organization and indexing schema), and Memory Processing (the mechanisms for creation, retrieval, updating, and forgetting). Formally, we conceptualize the memory scaffold as a dynamic state m t := ( object t , structure t ) m_{t}:=(\text{object}_{t},\text{structure}_{t}) , specifying the currently stored objects and their overarching organization. Driven by an execution-derived learning signal 𝒮 t \mathcal{S}_{t} (e.g., retrieval failures, task feedback, or capacity limits), memory evolution is formalized as: m t + 1 = IMPROVE ​ m ​ ( m t , 𝒮 ​ t ) . m_{t+1}=\text{IMPROVE}m(m_{t};\mathcal{S}t). (20) This update is instantiated through the memory processing module—a signal-driven family of operations (Write, Read, Update, Delete) parameterized by 𝒮 t \mathcal{S}_{t} that governs what to consolidate into object t + 1 \text{object}_{t+1} and how to reorganize structure t + 1 \text{structure}_{t+1} . Finally, it is crucial to note the scope of our discussion. While some literature considers knowledge internalized within the foundation model’s weights as "parametric memory" ( Wu et al., 2025d ; He et al., 2025 ; Zhang et al., 2025n ) , this section focuses on non-parametric, externalized memory embedded within the agent’s scaffold, maintaining the core assumption of a frozen foundation model.

#### 6.2.1 Memory Object

Recalling our formal decomposition of memory state, this subsection focuses on object t \text{object}_{t} , i.e., what is stored within the agent’s memory scaffold. The design of the memory object is paramount, as it directly governs storage efficiency, representation fidelity, and the capacity for cross-context knowledge transfer ( Koley, 2025 ) . Moving beyond raw, exhaustive interaction trails (e.g., infinite chat histories), self-improving agents increasingly favor storing processed, high-density abstractions ( Rasmussen et al., 2025 ; Deng et al., 2024 ) . By utilizing execution feedback ( 𝒮 t \mathcal{S}_{t} ) to filter or compress experiences, agents effectively mitigate context-window limitations and storage costs ( Cao et al., 2025 ) . These memory objects can be fundamentally categorized into explicit and implicit representations.

• Explicit Objects. Explicit objects are human-readable and directly manipulable, which makes them the default choice when interpretability, attribution, and safety auditing are important. Their main advantage is controllability: developers can inspect, correct, and selectively expose them to the model. Their main limitation is scalability, since verbose or weakly curated memories tend to increase retrieval noise and context pressure as the memory grows.

A first and widely used manifestation stores processed interaction trails that compresses raw trajectories into reusable, semantically meaningful units (e.g., distilled routines, heuristics, or reflections). Driven by task success/failure signals, agents abstract generalizable strategies from experience or maintain note-like intermediate traces to support long-horizon reasoning ( Wang et al., 2025i ; Ouyang et al., 2025b ; Lanchantin et al., 2023 ; Lee et al., 2024a ; Long et al., 2026 ) .

A second manifestation retains curated raw content . Certain scenarios require preserving exact surface details that are hard to summarize without loss (e.g., code snippets, formulas, or screenshots). Rather than storing everything, self-improving agents selectively write back only high-value artifacts validated during trial-and-error, distilling dynamic “cheatsheets” for future reuse ( Zhao et al., 2024a ; Suzgun et al., 2025 ) .

A third manifestation integrates external knowledge . Agents can integrate and maintain facts from external repositories. Grounding memory in shared references rather than free-form recollection enhances verifiability. Crucially, unlike static retrieval systems, self-improving agents utilize utility-based feedback to dynamically update, annotate, or prune these retrieved domain references (e.g., codebases or task-specific documents), thereby mitigating error propagation in downstream reasoning ( Tran et al., 2025 ; Zhang et al., 2024b ; Peng et al., 2023 ) .

• Implicit Objects. Implicit objects store memory in machine-native latent representations, including latent tokens, hidden states, and key–value cache augmentations. Their primary advantage is compactness and fast associative access, which makes them appealing under strict context limits or latency budgets. Their primary drawback is limited interpretability, which complicates debugging, targeted correction, and safety auditing, and can lead to representation drift when the memory is repeatedly rewritten or composed. Recent advancements explore several mechanisms for implicit memory scaffolding without altering the base FM parameters. Generative latent memory constructs latent sequences to enrich reasoning beyond text-based retrieval ( Zhang et al., 2025e ) . Latent state reconstruction captures and reintegrates hidden representations to improve context retention ( Dillon et al., 2025 ) . A related line augments the decoding process via offline coprocessors that inject latent embeddings directly into the KV cache to boost generation fidelity ( Liu et al., 2025b ; Sun et al., 2025a ) . Finally, maintaining self-updatable latent memory pools offers a practical compromise between persistent state tracking and strict capacity control ( Wang et al., 2024d ; Wang et al., 2025g ) .

##### Trade-offs.

Explicit memory objects offer high interpretability and controllability, simplifying debugging and safety auditing, but require rigorous curation to avoid overwhelming the context budget. Among them, processed trails favor generalization, curated artifacts favor precision, and integrated external knowledge favors verifiability. Conversely, implicit memory objects provide compact, high-speed associative access and mitigate context length issues, but are difficult to inspect, correct, and are susceptible to representation drift over long horizons. We summarize these qualitative trade-offs and common failure modes in Table 3 .

#### 6.2.2 Memory Structure

Building upon the definition of memory objects ( object t \text{object}_{t} ), the memory structure ( structure t \text{structure}_{t} ) specifies the organizational schema and relational topology through which these objects are indexed, linked, and retrieved. For self-improving agents operating in complex, dynamic environments, an adaptive memory structure acts as more than a passive database; it actively empowers the agent to consolidate long-horizon experiences, resolve context fragmentation, and perform robust relational reasoning ( Zeng et al., 2024 ) . Consequently, how this structure is designed and updated directly dictates the agent’s ability to scale its cognitive baseline efficiently. Prevailing structural paradigms can be broadly categorized into the following forms, each offering distinct trade-offs in scalability, expressiveness, and retrieval latency:

• Flat Structure. Flat memories store entries in strict temporal order. This append-only design makes writing computationally cheap and preserves the causal narrative of interaction. This topology is particularly advantageous for self-improving loops that rely on trajectory replay, as it maintains the unadulterated context necessary to diagnose failures ( 𝒮 t \mathcal{S}_{t} ) and reproduce successful behaviors. The primary cost is retrieval scalability. As the stream expands, recall becomes overly dependent on truncation heuristics or coarse filtering. Consequently, agents often suffer from recency bias—surfacing recent but irrelevant interactions while missing older, decisive evidence. Furthermore, lacking inherent support for abstraction, flat structures tend to accumulate redundant low-level traces, exacerbating context pressure over long horizons ( Antony et al., 2024 ; Sun and Zeng, 2025 ) . Representative systems operationalize this design by augmenting chronological logs with lightweight indexing. For instance, SCM maintains a temporal memory stream enriched with summaries and embeddings, bridging basic semantic search with chronological access ( Wang et al., 2025a ) . Similarly, frameworks like Self-Notes write transient insights directly inline during long-context reasoning. This keeps the memory aligned with the evolving cognitive state, allowing for immediate, feedback-driven course corrections while preserving the chronological flow of thought ( Lanchantin et al., 2023 ) .

• Hierarchical Structure. Hierarchical memories organize objects across multiple abstraction levels, addressing a central bottleneck in self-improving agents: the need to compress long histories into stable, reusable knowledge while preserving interaction-level details. By allocating high-level summaries, mid-level plans, and low-level traces into distinct layers, this topology reduces retrieval noise and supports long-horizon coherence. However, the primary risk is structural brittleness. If the induced taxonomy misaligns with the task, strict hierarchies can fragment evidence and hinder cross-cutting retrieval, which severely impedes the agent’s ability to recombine diverse experiences for continual improvement. Recent systems instantiate this hierarchy through different organizational lenses. For task-oriented abstraction, MobileGPT employs a goal-to-subtask-to-action hierarchy for GUI tasks, supporting both plan reuse and fine-grained execution recall ( Lee et al., 2024b ) . For semantic abstraction, H-MEM ( Sun and Zeng, 2025 ) and SHIMI ( Helmi, 2025 ) construct multi-level semantic nodes, enabling coarse-to-fine retrieval and top-down traversal from abstract intents to specific entities. Alternatively, SALM approaches hierarchy via tiered storage patterns (short-term, long-term) to separate active context maintenance from durable reuse ( Koley, 2025 ) . Besides, similar principles extend to multimodal domains, where agents separate transient perceptual buffers from persistent representations for long-video understanding, drawing inspiration from classical human memory models ( Cheng and Schwing, 2022 ; Song et al., 2024a ; Atkinson and Shiffrin, 1968 ) .

• Graph-Based Structure. Graph memories represent objects as nodes linked by semantic, temporal, or causal edges, aligning naturally with the need for self-improving agents to generalize across continuous interactions. By explicitly encoding relations, this topology replaces recency-based recall with retrieval by association. It enables multi-hop evidence aggregation that flat streams or strict trees struggle to support, which is vital when agents must attribute failures to latent dependencies, track evolving entity states, or reuse abstracted insights. However, the primary trade-off is maintenance complexity. Continuous graph construction, edge updating, and conflict resolution introduce significant computational overhead and risk structural drift as the agent repeatedly edits its own memory. Recent literature explores this paradigm through different relational lenses. For semantic and conversational tracking, systems like Mem0 ( Chhikara et al., 2025 ) and SGMem ( Wu et al., 2025e ) parse dialogues into fact- or sentence-level graphs to support highly structured recall. To support temporal and causal reasoning, Zep provides a temporal-aware knowledge graph for continuous belief revision ( Rasmussen et al., 2025 ) , while CausalRAG leverages explicit causal edges to prevent spurious correlations from contaminating the self-improvement loop ( Wang et al., 2025d ) . Some approaches even blend topologies; for instance, G-Memory introduces a hybrid hierarchical graph to separate reusable insights from fine-grained logs in multi-agent settings ( Zhang et al., 2025d ) . Beyond text, explicit graph structures are similarly crucial for multimodal agents—such as GraphVideoAgent and Scene-MMKG—where maintaining spatial-temporal state transitions and entity interactions is essential for grounded reasoning ( Chu et al., 2025 ; Song et al., 2024b ) .

• Vector Retrieval Structure. Vector-based memory indexes objects by dense embeddings and retrieves by similarity, which makes it a strong default when an agent must recall semantically related interactions under natural language queries. For self-improving agents, it serves as the primary engine for episodic recall and large-scale knowledge assimilation, supporting continual expansion without a hand-crafted schema. Its dominant failure mode is misalignment between similarity and usefulness. The nearest neighbors under an embedding metric may be topically similar yet decision-irrelevant, and this retrieval noise can systematically bias the agent’s subsequent learning updates. As a result, self-improving systems often augment vector retrieval with re-ranking, hybrid signals, or adaptive controllers that dynamically gate the retrieval process. Building upon the classic RAG pipeline ( Lewis et al., 2020 ) , Agentic RAG integrates autonomous control over retrieval to manage dynamic episodic memories ( Ravuru et al., 2024 ; Singh et al., 2025 ) . To mitigate the similarity-usefulness misalignment, systems explore various optimizations: hybrid heuristics , such as Generative Agents combining dense relevance with recency and importance ( Park et al., 2023 ) ; and adaptive representations , where RMM introduces differentiable ranking to improve recall under long-term dialogues ( Tan et al., 2025b ) . For domain-specific trajectory reuse, systems like MemoryBank ( Zhong et al., 2024 ) and CTIM-Rover ( Lindenbauer et al., 2025 ) index episodic segments to robustly fetch past successes, whereas MIRIX routes queries across multiple specialized sub-stores to overcome the limits of a flat index ( Wang and Chen, 2025 ) . Finally, this vector-based topology readily accommodates multimodal experiences, enabling embodied agents like MrSteve to retrieve relevant past video frames for exploration ( Park et al., 2025 ) .

#### 6.2.3 Memory Processing