Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering — part 2/11

Cleaned-text code points [15083, 30814). Bibliography omitted here; source.md includes it.

The Weights layer in Figures 2 and 1 corresponds to the earliest wave of modern LLM deployment, in which capability was identified almost entirely with model parameters. Pretraining on large corpora compressed broad statistical regularities, world knowledge, and latent reasoning habits into the weights ( Brown et al., 2020 ; Chowdhery et al., 2023 ; Touvron et al., 2023 ) . Scaling laws revealed predictable relationships between parameter count, data volume, and loss, reinforcing the intuition that progress tracked directly with model size ( Kaplan et al., 2020 ; Hoffmann et al., 2022 ) . By the time systems such as GPT-4 ( OpenAI, 2023b ) , Gemini ( Gemini Team et al., 2023 ) , DeepSeek-V3 ( DeepSeek-AI, 2025 ) , and Qwen2.5 ( Qwen Team, 2025 ) demonstrated broad multi-task competence, the dominant narrative in much of the field equated better agents with bigger, better-trained models. Supervised fine-tuning and preference optimization then shaped these models into more useful assistants by teaching instruction following, conversational style, refusal behavior, and domain-specific conventions ( Ouyang et al., 2022 ; Bai et al., 2022a ) ; direct preference optimization further simplified this alignment stage by eliminating the need for a separate reward model ( Rafailov et al., 2023 ) . From this viewpoint, improvement largely meant modifying or replacing the model itself.

This paradigm remains foundational, and weight-space capability offers several advantages: fast inference without external lookups, compact deployment, and strong generalization across many tasks without task-specific plumbing. The same model that answers a medical question can write a poem, debug a program, or summarize a legal contract, all without any change to the surrounding system. For one-shot, context-contained tasks, the weight-centric view is often sufficient.

However, weight-space encoding also couples knowledge, procedure, and policy too tightly to a static artifact. Updating a single fact—say, the current head of state of a country—requires retraining, knowledge editing ( Meng et al., 2022 ; Mitchell et al., 2022 ; Yao et al., 2023b ) , or patching through additional alignment layers, all of which risk unintended side effects on other capabilities. Auditing why a model behaved a certain way is difficult because relevant knowledge is distributed across billions of parameters rather than encoded as inspectable modules ( Zhao et al., 2024 ) . Personalization is also awkward: a single set of weights is asked to serve millions of users with different histories, preferences, and constraints, yet it has no mechanism to differentiate among them at the parameter level.

A central limitation of parametric knowledge is that it is difficult to selectively update, compose, and govern. As long as agents were confined to single-turn question answering, these weaknesses were often manageable. As systems moved into long-horizon task execution—where state accumulates, procedures must be followed reliably, and multiple tools must be coordinated—the difficulty of modularly managing knowledge, skills, and interaction rules inside the weights became more operationally salient. This shift encouraged developers to relocate some of these burdens into the next layer rather than relying on the model parameters alone.

### 2.2 Capability in Context

The Context layer represents the stage at which attention shifted from model modification to input design. Prompt engineering demonstrated that model behavior could be substantially altered without touching the weights: few-shot examples, role descriptions, chain-of-thought decomposition, and self-consistency traces all changed how the same model performed on the same underlying task ( Wei et al., 2022 ; Wang et al., 2023b ; Kojima et al., 2022 ) . Techniques for more structured reasoning soon followed. ReAct interleaved reasoning traces with tool actions in a single generation loop, showing that prompting alone could produce agent-like behavior without any architectural change ( Yao et al., 2023a ) . Tree of Thoughts generalized chain-of-thought into deliberate search over intermediate reasoning states ( Yao et al., 2024 ) . Self-Refine introduced iterative self-critique, demonstrating that models could improve their own outputs through multi-turn prompting loops ( Madaan et al., 2023 ) . Automatic prompt optimization further reduced the manual burden by using the model itself to search over the prompt space ( Zhou et al., 2023 ; Pryzant et al., 2023 ) . Retrieval-augmented generation (RAG) introduced a more systematic form of externalization by dynamically injecting external documents into the context at query time ( Lewis et al., 2020 ; Borgeaud et al., 2022 ; Ram et al., 2023 ; Gao et al., 2024 ) . Attention thus shifted from what the model had internalized to the information pipeline surrounding each invocation.

This stage made agent design substantially more flexible. Developers could attach local instructions, domain knowledge, output schemas, and retrieved evidence at runtime without any gradient update. Context became the medium through which developers staged cognition for the model—a working surface on which the right information could be assembled just before the model needed it. In many practical systems, iterating on prompts and retrieval pipelines proved substantially cheaper and faster than fine-tuning. The model could remain frozen while the surrounding prompt template, retrieval logic, and tool specification evolved rapidly.

The context-centric stage can also be interpreted through Norman’s notion of representational transformation. A difficult recall problem—“does the model know fact X X ?”—was converted into a recognition problem: “given that fact X X has been placed in context, can the model use it?” This resembles the recall-to-recognition shift associated with writing in the human externalization arc (Figure 1 , upper panel). The model did not need to have memorized the answer; it needed only to recognize and apply the relevant passage once it was provided. In Figure 2 , this transition corresponds to the emergence of prompting, RAG, chain-of-thought, and related techniques in the Context layer Zhang et al. (2024) ; Ram et al. (2023) .

Context-centric design also has important constraints. Context windows are finite, costly at scale, and often noisy when overloaded with marginally relevant material. Long prompts can degrade performance rather than improve it: the “lost in the middle” phenomenon shows that models attend unevenly across long inputs, with retrieval accuracy dropping sharply for information placed in the center of the context ( Liu et al., 2024a ) . Even as context lengths have expanded dramatically—from 2K tokens to over 100K and beyond ( Chen et al., 2023 ; Peng et al., 2024 ) —the fundamental tension persists: more capacity does not eliminate the need for selective curation. Context is also ephemeral: unless state is explicitly externalized elsewhere, every new session begins with partial amnesia. As systems become more complex, prompt assembly alone can become a brittle and ad hoc control mechanism. A model can be given more instructions, but that does not mean the system knows how to persist state across sessions, schedule multi-step workflows, coordinate among sub-agents, recover from partial failures, or enforce behavioral constraints over time. These limitations help explain the next outward step.

### 2.3 Capability through Infrastructure

The Harness layer—the topmost band in Figure 2 and the rightmost region in Figure 1 (lower panel)—represents the current stage, in which capability extends beyond prompt management into persistent infrastructure. As context windows became saturated and prompt templates more unwieldy, engineering attention increasingly shifted from “what should we tell the model?” to “what environment should the model operate in?” In mature agent systems, reliability increasingly depends on external memory stores, tool registries, protocol definitions, sandboxes, sub-agent orchestration, compression pipelines, evaluators, test harnesses, and approval loops ( Wang et al., 2024a ; Li, 2025 ; Luo et al., 2025 ; Xi et al., 2023 ) .

The earliest manifestations of this shift were simple but revealing. Projects such as Auto-GPT ( Richards, 2023 ) and BabyAGI ( Nakajima, 2023 ) wrapped an LLM in a loop with a task queue, persistent memory, and web access, showing that even a minimal harness could sustain behavior that no single prompt could. More principled frameworks quickly followed: AutoGen formalized multi-agent message exchange ( Wu et al., 2023 ) , MetaGPT added role-based collaboration and explicit procedures ( Hong et al., 2023 ) , CAMEL explored structured dialogue for task decomposition ( Li et al., 2023 ) , and Reflexion persisted feedback across episodes ( Shinn et al., 2023 ) . Across these systems, the common move was to shift burden out of the model and into surrounding structure.

The same move is now visible across deployment domains. Coding agents embed the model in development harnesses with files, shells, version control, tests, and reusable skill artifacts; SWE-agent and OpenHands are representative examples ( Yang et al., 2024a ; Wang et al., 2024b ) . Research and enterprise agents add retrieval, approvals, browsing, and long-horizon orchestration pipelines, as in Deep Research-style systems ( OpenAI, 2025b ; Google, 2024 ) . Embodied and workflow systems such as Voyager, LangGraph, CrewAI, and OS-Copilot likewise make control flow, environment access, and reuse explicit ( Wang et al., 2023a ; LangChain, 2024 ; CrewAI, 2024 ; Wu et al., 2024 ) . The recurring pattern is that reliability problems are increasingly solved by changing the environment rather than by prompting alone.

As shown in Figure 3 , the harness encompasses three major classes of externalization—memory, skills, and protocols—which correspond to the three major classes of burden that the harness absorbs. Memory systems externalize state across time, so that continuity no longer depends on ephemeral context. Skill systems externalize procedural expertise, so that complex workflows are loaded rather than reinvented. Protocols externalize interaction structure, so that tool and agent coordination follows governed contracts rather than ad hoc prompting. Together, these elements make up the harness: the persistent infrastructure that envelops the model and transforms the tasks it faces into forms that its internal competencies can handle more reliably.

Under this framing, “agent engineering” increasingly takes the form of “harness engineering.” The model remains the core reasoning engine, but it is no longer the sole location of intelligence. Capability is distributed across the structures that shape what the model sees, remembers, calls, and is allowed to do.

### 2.4 Externalization as the Transition Logic

Taken together, the path from weights to context to harness is a story of externalization in Norman’s sense ( Norman, 1991 ) : burdens that are hard to manage inside the model are progressively moved into explicit artifacts outside it, and the task seen by the model is correspondingly transformed. Mutable knowledge moves from weights into retrieval systems and runtime context, converting recall into recognition. Reusable procedures move from implicit habits into explicit skills, converting improvised generation into structured composition. Interaction rules move from ad hoc prompting into protocols, converting ambiguous coordination into governed contracts. Runtime reliability, in turn, moves into harness logic, where constraints, observability, and feedback loops can be made explicit.

This redistribution is best understood as a response to mismatch. LLMs are strong at flexible synthesis and reasoning over provided information; they are less reliable at stable long-term memory, procedural repeatability, and governed interaction with external systems. Externalization therefore constructs a larger cognitive system around the model rather than replacing it, a view consistent with cognitive-architecture accounts such as CoALA ( Sumers et al., 2024 ) . The three harness dimensions follow directly from this framing: memory addresses continuity over time, skills address consistency of procedure, and protocols address structure of interaction. The following sections examine each in detail.

## 3 Externalized State: Memory

Memory externalization addresses the temporal burden of agency. A bare language model must carry continuity, prior experience, user-specific facts, and partially completed work inside an ephemeral prompt. Once tasks extend across sessions, branches, or interruptions, that burden becomes both unstable and expensive. Memory externalizes it into persistent state that can be written, updated, and retrieved outside the model.

In harnessed agents, memory is more than an archive. It supplies checkpoints for resumable execution, traces from which skills can be distilled, statistics that influence protocol routing, and persistent state that governance mechanisms can inspect and constrain. To make that role precise, this section asks three linked questions: what burden memory externalizes, how the design space has evolved, and how memory couples to the broader harness. Section 3.1 clarifies which kinds of state are externalized; Section 3.2 surveys the main architectural choices; Section 3.3 turns to the demands imposed by harnessed agent systems; and Section 3.4 closes the chapter by interpreting memory through the lens of cognitive artifacts.

### 3.1 What Is Externalized: The Content of State

The essence of memory lies in decoupling the agent’s state across time from its transient context. The relevant contents are not every external artifact in the harness, but the records that preserve continuity: current task state, past execution experiences, abstracted knowledge, and persistent user or environment context. To maintain coherent behavior across long-horizon interactions, the memory system must categorize and manage these records according to their temporal properties and retrieval needs. Drawing inspiration from classical taxonomies of human memory and adapting them to LLM agents, we distinguish the following four dimensions of externalized state:

##### Working context.

Working context is the live intermediate state of the current task: open files, temporary variables, active hypotheses, partial plans, and execution checkpoints. It changes quickly and loses value if it is stale, but without externalization it disappears as soon as the context window resets or a process is interrupted. Coding agents illustrate the point well. By materializing drafts, terminal state, and workspace artifacts outside the prompt, systems such as OpenHands and SWE-style agents can resume from the current operating state rather than reconstructing it from scratch ( Wang et al., 2025a ; Yang et al., 2024b ) .

##### Episodic experience.

Episodic experience records what happened in prior runs: decision points, tool calls, failures, outcomes, and reflections. Its value is not merely archival. Retrieved episodes can serve as concrete precedents, help the agent avoid repeating known mistakes, and supply raw material for later abstraction. Reflexion made this pattern explicit by storing reflective summaries from failed attempts as reusable experience ( Shinn et al., 2023 ) . AriGraph extends the idea further by treating local interaction trajectories in unfamiliar environments as episodic memory from which a broader world model can be built ( Anokhin et al., 2024 ) .

##### Semantic knowledge.