MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory — part 6/6

Cleaned-text code points [78041, 81243). Bibliography omitted here; source.md includes it.

Beyond this collective pooling, a significant and promising frontier involves investigating the selective nature of such knowledge diffusion through the lens of transfer learning. As the number of agents increases, particularly beyond simple pairings, the challenge of ”what to share” and ”with whom to share” becomes a critical research direction. Future work could explore mechanisms that differentiate between universally applicable procedural insights and task-specific noise, ensuring that memory transfer remains contextually relevant and avoids the risks of negative transfer. Navigating these trade-offs between global collective intelligence and targeted knowledge distribution will be essential for building scalable, self-evolving swarms.

### G.6 Dedicated Domains and Hybrid Architectures

While this work strongly advocates for freezing large language models (LLMs) to prevent catastrophic forgetting, a question arises regarding highly specialized domains where the base model may not comprehend foundational terminology. Is MemRL alone sufficient in such cases? We anticipate a future hybrid model where companies periodically fine-tune foundational models (e.g., annually) to update their core vocabulary and understanding, while simultaneously leveraging MemRL for daily behavioral adaptation and runtime learning. This approach requires the base model to possess a relatively high level of “intelligence” to initially grasp and subsequently learn new terms through interaction and feedback.

## Appendix H Case Study: High-Utility Failure Analysis (Near-Misses)

This appendix section provides qualitative case studies of high-value near-miss memories mined from the 10-epoch OS-Interaction run. CS denotes Case Study . Each box below contains: origin task, retrieved reflection memory, a short explanation, and the target task where it was retrieved.

## Appendix I Prompts

We provide the exact prompt strings and message templates used by our MemRL implementation across all benchmarks. To minimize ambiguity, we separate prompts used to summarize experiences into memories from prompts used at task time for generation/inference.

### I.1 Experience Summarization Prompts

### I.2 Generation and Inference Prompts

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .