When Does Continual Learning Require Learning — part 2/4

Cleaned-text code points [15624, 31182). Bibliography omitted here; source.md includes it.

Architectural / Context Compression (Cartridges, In-place TTT) These methods change the model’s update interface rather than the loss applied to its parameters. Cartridges [ 10 ] freezes the backbone and learns a small per-stage component, distilled from a teacher trace and stitched in at inference. The backbone is θ 0 \theta_{0} for every stage; each stage’s update is structurally local. In-place TTT [ 11 ] performs an input-conditioned weight update at inference using a self-supervised auxiliary loss, reset between inputs. We include In-place TTT as a locality reference: its update is per-input rather than per-stage, which lets us isolate the locality axis from the accumulation axis in the experiment.

## 4 Experiments

For each task evaluation, we apply our general continual learning protocol and vary only how the world changes between optimization stages. We work with the two axes of change: space , where the model moves across substantively different domains, and time , where the model stays in one domain but the underlying data drifts across stages. Within these axes we instantiate four settings: independent domains (space), where each stage is a different task; selective fact change (time), where some labels change between stages while others remain stable; continuous temporal drift (time), where the task is fixed but the data slides smoothly across stages; and and agentic accumulation (time), where the task family is fixed but the environment’s state drifts as a direct consequence of the agent’s own prior actions rather than an external clock or editor. More details on task design choices can be found in Appendix A.2 .

### 4.1 Domain Transfer

When a model is exposed to a new domain, it will need to acquire new skills or capabilities without catastrophically forgetting. This is our evaluation setting and the classic test of continual learning and covers the axis of space.

Setup. Three unrelated supervised tasks are presented in a fixed order — ToolUse [ 50 ] , FinQA [ 5 ] , SciKE-Bio [ 12 ] with each phase training a Qwen-3-8B on a 500-example subsample of one task’s training set and validating against all three tasks at every step. The subset of ToolUse and SciKE-Bio are taken from [ 48 ] . The task chain produces a forgetting matrix that answers "after seeing tasks 1 ​ … ​ i 1...i , how well does the model still do task j j ?" The three tasks deliberately share no surface-level structure, so a method that nominally improves task acquisition can still fail by drifting the model’s output format or by overwriting earlier formats once it learns a later one. For continual training, the best performing checkpoint on the current task is carried over to the next one. Further details about the experimental setup including relevant hyperparameters are included in Appendix A.2.1 .9

Prompt and harness optimization surfaces a speed-vs-stability trade-off. We observe that GEPA fits very quickly to some training stages such as FinQA, but has the most severe catastrophic forgetting when moving on to the next stage. In Fig. 2 we observe that despite an almost 40% increase in performance during the FinQA training stage, after the training on biology accuracy drops almost entire back down the baseline accuracy before any training. While ACE appears to be slight more stable than GEPA, it similarly improves and then immediately degrades at this same phase transition.

Distillation-based methods manage to accumulate knowledge. We observe that SDFT and SDPO both finish the sequential training with FinQA and Bio above their zero-shot baselines while keeping ToolUse near 55–60%; SDFT is the only method in our table whose three final-stage scores are all at or above their respective zero-shot levels.

Context compression does not help learn new tasks. Cartridges and In-place TTT are less sensitive to than prompt methods to domain shift, but at the same time they do not show steady improvement across all tasks over training phases like SDFT. Performance is largely flat or even simply degrades per task at each progressive training phase. These methods never perform a full weight update, and thus appear to struggle to accumulate new capabilities under the domain shift task.

### 4.2 Catastrophic Memorizing: Update Truly New Knowledge

In the previous experiment, we tested methods on their ability to stay aligned and accumulate knowledge across time. Here, we ask the opposite: when presented with discrete fact changes, can a model rewrite the affected belief while leaving the rest of its knowledge untouched? Avoiding catastrophic forgetting is necessary, but so is knowing when and how to update. The failure to do so is what we call ’catastrophic memorizing’.

Data. We construct the chain from four monthly Wikipedia mirrors covering the post-pretraining-cutoff window: 2025-11-20, 2025-12-01, 2026-01-01, and 2026-02-01. For each adjacent pair of snapshots we extract a drift set of (subject, relation, object) triples whose object value changed between the snapshots, treating these as new world-knowledge facts the model is being asked to acquire. We also extract a stable set of (subject, relation) pairs whose object did not change across the entire window, and hold this set out across all phases as a forgetting probe: a method that improves drift accuracy by overwriting the model’s general world knowledge will pay for it on the stable probe.

Setup. To test for small volume adversarial updates evolving over time on Qwen-3-8B, Wikipedia is mirrored at four monthly snapshots, where every method updates from the same Wikipedia diff snapshot, in its method-appropriate format. Weight-update and RL methods (SFT, SDFT, SDPO, GRPO) consume the 500 drift triples directly as Q/A pairs. Cartridges and In-place TTT consume the corresponding Wikipedia article bodies pulled from per-snapshot – approximately 528 articles per slice. Further details about the experimental setup including relevant hyperparameters are included in Appendix A.2.2 . A filtered, continuously-scored variant of this benchmark (TempWiki-Easy), which isolates the update behavior from noisy diff categories and binary scoring, is analyzed in Appendix A.2.3 .

Stability-anchored methods fail to learn how to change. Interestingly, SDFT and SDPO, which perform well on accumulating knowledge in noisy signal, struggle to learn changes in this setting. SDFT’s stable-fact F1 falls from ≈ \approx 0.30 0.30 at s 1 s_{1} to ≈ \approx 0.15 0.15 at s 3 s_{3} : by trying to interpolate the new slice against its own previous beliefs, the model ends up corrupting facts that were never supposed to move. SDPO shows the same effect at smaller magnitude. In contrast, GRPO shows strong performance in learning how to change while preserving performance averaging above baseline across the chain + 1.6 +1.6 points. GRPO witnesses the sharpest jump at s 2 s_{2} of + 6.2 +6.2 points which is the largest measured gain of any weight update method.

Prompt evolution shows strong optimization ability on adapting to new knowledge at the cost of forgetting stable knowledge. GEPA reaches the highest drift F1 in the comparison (0.18 on s 1 s_{1} , 0.24 on s 3 s_{3} ) by writing the new fact directly into the playbook in a single optimization step. The optimizer is rewarded on the validation set, which contains drift facts; it is not rewarded on stable facts, which it never sees. So GEPA edits the playbook freely, and the same edits overwrite stable entries that depend on the same playbook tokens. Stable F1 falls from 0.32 at s 1 s_{1} to 0.26 at s 3 s_{3} . ACE shows the same profile although at a smaller magnitude.

Context compression barely influences the base model. For cartridges and In-place TTT drift F1 stays within a few points of baseline across all three slices, with no separation in either direction. Cartridges averages − 0.8 -0.8 on drift, with one + 3.2 +3.2 inflection on s 2 s_{2} and despite performing better than SDFT and SDPO, but shows no improvement compared to baseline, even harming the performance, belonging to the same category at times.

### 4.3 Noisy Temporal Drift

From the previous experiments, we observe over longer time horizons, performance steadily degrades. In our following temporal drift experiments, we dig deeper into the problem by setting up a task where the model can only improve if it manages to accumulate weak and noisy signals through time.

Setup. We construct a challenging temporal data task by taking a realistic case study in finance: predicting whether a stock moves up or down in the 30 days after its 10-K filing. This setting forces a model to operate inside a two-sided time drift. Some patterns in how firms describe risk, growth, and capital plans should be retained, while other signals are regime-specific (e.g. before and after Covid) and stop working as markets evolve. Doing well now does not guarantee doing well in the future. A model that loses past knowledge cannot tell whether a new filing is unusual relative to history, and a model that holds too tightly to past years may overfit on spurious or transient regimes.

We use full-length SEC EDGAR filings (typically 10–13K tokens), with each filing labeled up or down based on its 30-day forward return [ 34 , 37 ] . Additional details on data can be found in Appendix A.2.4 . We train sequentially on filing years 2015 through 2020 in chronological order. After each year, we evaluate on every year in the range to measure both backward transfer (did the model actually learn during training) and forward transfer (did it learn something general it can apply to future years). The signal in any individual filing is known to be weak, but in aggregate a small, tradeable edge can be accumulated [ 37 ] (see also Figure 8 ). At baseline before training, Qwen3-8B sits around 50% and accuracy is noisy across years. Further details about the experimental setup including relevant hyperparameters are included in Appendix A.2.4

RL struggles when the reward signal is weak and noisy. GRPO shows a significant accuracy drop both on forward- and backward-looking sentiment; its success depends heavily on the reward ranking inside each group. If the reward is noisy, the algorithm may reinforce the wrong answer.

Stability-anchored methods show the strongest forward transfer with minimal past degradation. SDFT’s past-year accuracy stays flat around 0.50 0.50 while its future-year accuracy climbs to ≈ \approx 0.62 0.62 by 2020: distilling against its own previous state avoids both drift and over-fitting to the current year. Cartridges is the most stable method we tested, with past and future hovering around 0.55 0.55 – 0.58 0.58 throughout the chain. In-place TTT is the locality reference: its accuracy stays in a tight 34.5–51.5% band across the chain (mean 46.9%) and never separates from baseline. The reset-per-input update geometry is incompatible with absorbing a slow-moving trend, but it pays no price for trying – the method neither acquires nor degrades.

Prompting methods fail to extract useful knowledge for forward transfer. On this task, similar to RL, GEPA also struggles with learning meaningful information from noisy signals. ACE fits better on past-year data, but also completely fails to predict future years’ trend, which suggests that it does not accumulate meaningful knowledge.

Qualitatively, GEPA discovers generic financial-analysis heuristics (Figure 5 ). These edits fail to capture characteristics of an ideal adaptive analyst e.g. learn to discard irrelevant sections, normalize sentiment relative to sector or company baselines, or distinguish between newly informative elements and those that are already priced in by the market. While it is possible to construct a strategy with a positive Profit and Loss (PnL) with a rule as simple as focusing on relevant sections in the 10-K (see Figure 8 ), consistent with the observed drop in forward accuracy after optimization, the optimized prompt reduces even this forward PnL. We conclude that in this noisy, time-changing regime, GEPA overfits to transient textual heuristics while failing to preserve exposure to economically meaningful signals that remain predictive out-of-sample.

### 4.4 Agentic Task

The previous two experiments test the time axis under drift that comes from outside the model. Here we test a third sub-regime of time introduced in § 3 : the task family stays fixed (actions inside a web app), but the environment’s state drifts only because the agent itself acted on it, so state left behind by one step becomes part of the task the model faces at the next.

Data. We build the agentic suite on top of WebArena-Infinity [ 61 ] , seeding chain generation from each app’s description (which enumerates the available features) and its underlying state files (which fix the entities those features can act on). From these we generate sequential chains in which step i + 1 i{+}1 depends on a concrete piece of state that step i i leaves behind in the environment: in Gmail, create a label, rename it, then apply the renamed label to a specific email. Each step is paired with a programmatic verifier that reads the app’s state directly, so chain success is checked against the environment rather than the agent’s narration. Because the sequence arises from the agent’s own actions rather than an ordering we impose, we treat it as the most realistic instantiation of continual learning over time.

Setup. Beyond zero-shot evaluation, we ask whether experience compounds across a chain: whether learning from earlier trajectories improves later chain success. We add learning two ways on top of a browser-use agent driving a headless Chromium against the live app. First, an ACE playbook [ 59 ] on Qwen-32B, which accumulates reusable procedural guidance distilled from prior rollouts. Second, supervised fine-tuning of Qwen-8B on successful traces from a stronger teacher agent, distilling the larger agent’s behavior into the smaller model (per-level teachers are given in Appendix A.2.5 ). Within a chain we keep a single agent instance alive across all steps: the browser session is not reset, and each follow-up instruction is appended to the same conversation rather than starting a fresh context, so the model sees its own prior trajectory when acting on step i + 1 i{+}1 . Each step is given a fixed action budget and is scored independently by its verifier. Further details about the experimental setup including relevant hyperparameters are included in Appendix A.2.5 .

Results. Both update mechanisms raise end-to-end chain success over their no-learning baseline at every horizon length (Figure 6 ). The ACE playbook lifts Qwen-32B over its zero-shot baseline at every length, most on the three-step (L3) chains ( 32.4 % 32.4\% to 46.5 % 46.5\% ). Without training, Qwen-8B collapses to zero past L1; supervised fine-tuning on the teacher’s traces recovers substantial success ( 40.0 % 40.0\% at L3, 20.0 % 20.0\% at L5, and 6.7 % 6.7\% even at L10), and overtakes both Qwen-32B lines at L1 ( 60.0 % 60.0\% ). Absolute success still decays with chain length for every configuration, as accumulated trajectory state grows and later steps must be grounded in increasingly distant environmental cues, converging near the floor by L10. This is the closest setting to deployed use, since the chain comes from the agent’s own actions rather than an ordering we picked, and future work could go deeper on this realistic agentic setting.

## 5 Discussion

### 5.1 Conclusion