When Does Continual Learning Require Learning — part 3/4

Cleaned-text code points [31182, 47012). Bibliography omitted here; source.md includes it.

We introduced a unified framework for evaluating continual learning in large language models, defining it as the problem of increasing competence as the world changes along two axes: space (domain shift) and time (temporal drift). By recasting widely used LLM benchmarks as sequential problems and evaluating eight methods under a common mechanism-agnostic protocol, we surface a consistent set of tradeoffs that span across method families. No single method handles all task regimes well: prompt-based methods fit quickly but overwrite prior capabilities; distillation-based methods accumulate knowledge stably but resist new updates; context compression improves efficiency without improving task acquisition; and online reinforcement learning adapts most effectively to factual change but degrades under noisy reward signals. Overall, our results suggest that continual learning is not a single capability: different patterns of environmental change require fundamentally different update behaviors. We hope that understanding where each method succeeds and fails will help guide the design of more capable continual learning systems.

### 5.2 Limitations and Challenges

To draw unified conclusions, we use a single model across all evaluations, Qwen3-8B; the relative behavior of the tested methods may change for larger models or models in reasoning mode. In addition, our benchmark suite captures only a subset of realistic environmental change: domain shifts, agentic sequences, noisy temporal drift, and discrete factual updates. While this is a step beyond fixed benchmarks and already reveals that different regimes require different update behaviors, deployed LLMs will face many more open-ended, changing, and uncertain environments and diverse use cases. The results should therefore be read as evidence about which update behaviors are required in different regimes, not as a definitive ranking of continual-learning algorithms.

## Acknowledgement

We thank Matei Zaharia and Alyosha Efros for helpful discussions and feedback. This work is partially supported by the Laude Institute, Modal, and the ONR MURI N00014-21-1-2801. AH was supported by the National Defense Science and Engineering Graduate (NDSEG) Fellowship Program. This material is based upon work supported by the Air Force Office of Scientific Research under award number FA9550-25-C-B010 in the amount of $31,607.75.

## Appendix A Additional Experimental Details

### A.1 Compute and Hyperparameters

Within each benchmark, every weight-update method (SFT, SDFT, SDPO, GRPO) uses the same global batch (32), one epoch over 500 examples per phase ( ≈ \approx 16 optimizer steps), and full-weight updates. The outer learning rate is set per-method-per-benchmark to keep cumulative parameter displacement under the empirical collapse threshold for that label regime [ 8 ] . Each training phase took no longer than 1 hour on a node of 8 H100s.

Cartridges and In-place TTT use their own optimization recipes; hyperparameters are reported per benchmark in the sections below.

For prompting-based methods, we used cloud API models via OpenRouter. The total cost of all experiments (including development and hyperparamter tuning) was $1000 USD.

### A.2 Task Settings

#### A.2.1 Domain shift

The learner specialises, in sequence, across three substantively different domains: tool use (ToolUse [ 50 ] ), finance (FinQA [ 5 ] ), and biology (SciKE-Bio [ 12 ] ). Each stage provides 500 training instances and a 50-row held-out evaluation set per task; stages are presented in the canonical order [ToolUse, FinQA, Bio] so that per-method results are directly comparable, with permutation-averaged variants reported in the appendix to control for order effects.

Tasks and data. Each phase trains on a 500-example subsample of one task’s training set and validates against all three tasks at every step. The FinQA test set is unusually large (1,125 prompts); we deterministically subsample it to 128 prompts using the SDPO baseline’s seed ( np.random.RandomState(1042).permutation(1125)[:128] , sorted), so every method evaluates on identical FinQA prompts.

Per-method hyperparameters. SFT, SDFT, SDPO, and GRPO share the optimization budget in Table 2 . SDFT, SDPO, and GRPO additionally use 8 on-policy rollouts per training prompt (temperature 0.6, top- p p 0.95). SDPO optimizes a DPO-style pairwise objective over scored rollouts (chosen = high-score, rejected = low-score) against a frozen reference model. GRPO uses PPO-style clipped objective with per-prompt z z -scored advantages over the 8-rollout group.

Cartridges configuration. Each phase synthesizes a self-study corpus from the cumulative train pool of phases 1.. i i : Qwen3-8B (running locally as a synthesis server) is prompted with the four standard seed prompts ( use_case , question , summarization , structuring ) to emit 8,192 conversation-style Q/A pairs grounded in the corpus. Those 8,192 conversations are then context-distilled into a fresh trainable KV cache. Base weights are frozen. At evaluation the cartridge KV is loaded as a fixed prefix and the validation prompt is generated against it.

##### In-place TTT configuration.

Continual pretraining via VeOmni plaintext-iterable JSONL with one record per training prompt (cumulative pool of phases 1.. i i in order B). Fast-weight updates are activated on a sparse layer set (paper-recommended for continual training). The chain hands each phase’s HF-format checkpoint forward as the next phase’s model_path .

Scoring. Each task has its own scorer. A prediction in the wrong format scores zero — the benchmark measures task knowledge and format-fidelity together. • Tool-use: regex-extract every Action: <name> and Action Input: {...} line; parse each Action_Input body as JSON; accept iff the multiset of predicted action names equals the gold and the merged Action_Input dict equals the gold dict key-by-key.

• FinQA: extract the contents of the last \boxed{...} ; accept iff | pred − gold | ≤ max ⁡ ( 0.005 ⋅ | gold | , 0.01 ) |\text{pred}-\text{gold}|\leq\max(0.005\cdot|\text{gold}|,0.01) (FinQA paper convention; exact-match would penalize harmless rounding).

• SciKE-Bio: look for a strict <answer>X</answer> tag; accept iff the single-letter X equals the gold letter.

#### A.2.2 TempWiki Benchmark Details

Task and data. Wikipedia is mirrored at four monthly snapshots — 2025-11-20, 2025-12-01, 2026-01-01, 2026-02-01 — and for each adjacent pair we extract the drift set of (subject, relation, object) triples whose object value changed across snapshots. We also extract a stable set of (subject, relation) pairs whose answer did not change across the chain, and we hold this set out across all phases as a forgetting probe.

Using these snapshots, we construct a chain with three phases corresponding to three diff-pair slices, with 500 training drift triples, 50 validation drift triples, and 50 stable validation triples. Existing temporal-knowledge benchmarks such as the original TemporalWiki [ 21 ] stop at facts that have changed and do not audit unchanged knowledge, which makes the catastrophic-memorization failure mode invisible. Pairing the drift set with a held-out stable probe is what lets the matrix expose the trade-off methods make between acquiring the new fact and preserving the surrounding knowledge.

Per-method hyperparameters. SFT/SDFT/SDPO/GRPO use the recipe from the 500 drift triples Q/A-formatted directly; stable triples are held out and never appear in training. Cartridges uses the same configuration as on the domain benchmark, with two adjustments: the source corpus is the cumulative Wikipedia article bodies rather than the cumulative training pool, and the slice-1 article pool deliberately includes the stable-val subjects so the synthesizer has reason to write Q/A about them; only drift articles are added on later slices, which keeps the stable probe legitimately held out from the gradient-equivalent updates. In-place TTT uses the same fast-weight recipe as on the 10-K benchmark; per-article records are formatted as === Wikipedia article title=<sitelink> as_of=<snapshot> ===\n<body>\n followed by an optional Known facts: block holding the canonical subject relation -> gold lines.

Scoring (word-level F1, paraphrase-permissive). The scorer computes word-level F1. The prediction is stripped of <think> traces and chat suffixes; the model’s answer is taken to be the contents of an explicit Answer: ... line if present, otherwise the first non-empty line. Normalization (applied identically to prediction and gold): lower-case, strip the articles a , an , the , strip all ASCII punctuation, collapse whitespace, tokenize by whitespace into a token bag.

We score predictions with a token-level F1 against the gold answer. Both strings are normalized (lowercased, articles and punctuation stripped, whitespace collapsed) and split into tokens. Let S S be the number of tokens shared between the prediction and the gold (counted with repetition). Then P = S N pred , R = S N gold , F 1 = 2 ​ P ​ R P + R . P=\frac{S}{N_{\text{pred}}},\qquad R=\frac{S}{N_{\text{gold}}},\qquad F_{1}=\frac{2PR}{P+R}. A prediction counts as a hit if F 1 ≥ 0.5 F_{1}\geq 0.5 , and zero otherwise. We report score_mean@8 : the average hit rate across 8 rollouts per prompt and across all prompts in a slice.

#### A.2.3 TempWiki-Easy: a filtered, continuously-scored variant

The TempWiki drift set in the main text (Fig. 3 ) pools every relation type whose object value changed between two snapshots. Two properties of that construction make the catastrophic-memorizing signal noisier than it should be. First, many Wikipedia diffs are not genuine world-knowledge updates: reformatting, disambiguation edits, list re-orderings, and annotation-style relations change the object string without there being a well-posed “new fact” to learn. Second, the object-F1 scorer used there is effectively binary at the string level, so a near-correct object (e.g. a partial or alternately formatted name) is scored identically to a wholly wrong one. Both effects inflate the apparent divergence between drift and stable curves. TempWiki-Easy is a cleaned re-cut of the same benchmark designed to isolate the update behavior from these artifacts.

Construction. We keep the four monthly snapshots and the identical three-slice chain as TempWiki (Nov → \to Dec 2025, Dec 2025 → \to Jan 2026, Jan → \to Feb 2026), and apply three changes to the diff extraction: • Category allow-list. Only (subject, relation) pairs drawn from an allow-list of relation types for which “change” is well-defined (single-valued, factual, verifiable object) are eligible. Relations prone to formatting or annotation churn are excluded outright, which removes the pathological categories responsible for spurious drift/stable divergence in the original cut.

• Aggressive judged filtering. Each surviving candidate diff is passed through an LLM judge that keeps the triple only if the edit reflects a semantically meaningful factual update between the two snapshots, discarding cosmetic or ambiguous edits.

• QA regeneration + continuous scoring. We regenerate question/answer pairs over the filtered facts and, rather than thresholding the word-level F1 into a hit at F 1 ≥ 0.5 F_{1}\!\geq\!0.5 (the score_mean@8 hit rate used in Appendix A.2.2 ), report the continuous word-level F 1 F_{1} directly. Partially correct objects receive partial credit, so small real movements in knowledge are visible rather than being quantized to 0 / 1 0/1 .

The most visible difference from TempWiki Fig. 3 is that the stable-fact trajectory (purple dashed) no longer trades off against drift. In the original cut, SDFT’s stable F1 fell from to and SDPO’s collapsed to by the final slice; in TempWiki-Easy both remain flat and essentially coincident with the zero-shot stable baseline across all three slices. With the confound removed, most methods sit on top of the zero-shot drift baseline: SFT and SDFT neither learn the changed facts nor harm the stable ones. The two exceptions are the reinforcement/distillation updaters. SDPO climbs to 0.22 on the final slice, above its 0.17 zero-shot baseline, while its stable F1 drifts only mildly GRPO shows the largest net effect: after the same mid-chain dip it reaches ∼ \sim 0.22 drift on Jan → \to Feb while holding stable F1 flat.

#### A.2.4 Temporal Drift

10-K Sentiment Benchmark Details

The core task is fixed (predict the directional sentiment of a 10-K filing) [ 34 , 37 ] but slides across calendar time: each stage corresponds to one fiscal year of 10-K filings, presented in chronological order from 2015 to 2020. After training on year y y we evaluate on every other year, splitting the score into past-year accuracy ( ≤ y \leq y ) and future-year accuracy ( > y >y ). The target is intentionally weak-signal: real financial text is informative only in aggregate, and the question is precisely whether a learner can stay extract rules to align with a slow-moving, noisy trend rather than rely on spurious local-in-time patterns.

A second temporal benchmark constructed from Wikipedia revisions captured between November 2025 and February 2026 [ 21 ] . Stages correspond to three chronological slices ( s 1 , s 2 , s 3 s_{1},s_{2},s_{3} ) of facts whose object value changes over the window. We evaluate object-level F1 on each slice after every stage, together with F1 on a control set of stable (unchanged) facts. This exposes a failure mode invisible in standard CL benchmarks: methods that successfully update drifted facts may simultaneously degrade unchanged ones —the mirror image of forgetting.

Task and data.

Specifically, we construct a six-year chain of 10-K filings drawn from SEC EDGAR [ lefteris_loukas_2021_5589195 , 24 ] , covering fiscal years 2015 through 2020 in chronological order. Each phase contains ∼ \sim 500 training filings and 50 held-out validation filings drawn from that year, with each filing represented in full (typically 10–13K tokens) wrapped between custom [START OF FILING] / [END OF FILING] markers. Cumulative training tokens grow from ∼ \sim 6M after phase 1 to ∼ \sim 37M after phase 6. The label is a single token, up or down , derived from the issuer’s 30-day forward simple stock return joined to the filing by Central Index Key (CIK) and filing date. Prior work on financial NLP either treats each year as an independent task or pools all years; we hold the task fixed and let the underlying data-generating process drift, exposing methods to the actual condition under which deployed financial models age out of distribution.

In this task,the model predicts the directional sign of an issuer’s 30-day forward simple stock return from the textual content of its annual 10-K filing. The chain has six sequential phases for filing years 2015 through 2020 in chronological order, producing a 6 × \times 6 matrix. Each filing is full-length SEC EDGAR text wrapped between custom [START OF FILING] / [END OF FILING] markers (typically 10–13K tokens); the gold label is a single token, up or down , derived from a precomputed returns table joined by issuer CIK and filing date.

Per-method hyperparameters. SFT/SDFT/SDPO/GRPO use the optimization recipe from Table A.1 with two finance-specific data settings: data.max_prompt_length = 16,384 to fit the 10-K body, and data.max_response_length = 256 since the response is a single token plus chat tail.

Cartridges configuration. Per phase, concatenate the cumulative pool of all filings seen so far (years 1.. i i , shuffled at filing granularity); synthesize 8,192 self-study Q/A pairs grounded in that pool; distill into a fresh 2,048-token KV cache. The cartridge is loaded as a prefix at evaluation, with the full filing body still included in the user message.