When Does Continual Learning Require Learning — part 4/4

Cleaned-text code points [47012, 57226). Bibliography omitted here; source.md includes it.

In-place TTT configuration. Continual pretraining on the same cumulative corpus, written as VeOmni plaintext-iterable JSONL with one record per filing (full body). Fast-weight updates are activated on a sparse layer set (paper-recommended for continual training). The chain hands each phase’s converted HF checkpoint forward as the initial weights for the next phase.

Scoring. The prediction string is stripped of any <think> trace and chat suffixes, then matched against the case-insensitive whole-word patterns \bup\b and \bdown\b . The first match wins; ties break in favor of up . A prediction is correct iff the matched token equals the gold.

Further Details on the 10K sentiment task with GEPA

PnL computation. We construct a daily long-short strategy from model predictions. For each filing observed on day t t , the model predicts a directional position p i ∈ − 1 , + 1 p_{i}\in{-1,+1} , corresponding to DOWN or UP sentiment respectively. Let r i r_{i} denote the realized 30-day return following the filing. Daily portfolio returns are computed as the average position-weighted return across all filings observed on that day R t = 1 N t ​ ∑ i = 1 N t p i ​ r i R_{t}=\frac{1}{N_{t}}\sum_{i=1}^{N_{t}}p_{i}r_{i} , where N t N_{t} is the number of filings on day t t . The cumulative PnL is obtained by cumulatively summing the daily returns R t R_{t} over time. The optimized PnL curve is causal. After optimizing the prompt on year X X , predictions generated using the optimized prompt are evaluated only on year X + 1 X+1 , reflecting a sequential deployment setting. Figure 8 shows that it is possible to construct a strategy with a positive PnL pre-Covid through a simple rule: attend only to a subset of sections. Similar to the observed accuracy drop in Figure 4 , GEPA’s optimisation fails to generalise in a causal forward-looking manner, dropping PnL performance.

Section-wise sentiment with possibility to abstain We additionally explored a setup in which the model was allowed to abstain from a prediction for a given filing sections during GEPA optimization, as not all sections are considered equally informative. Figure 9 reports the relative change in section utilization frequency under this setup. Some sections became increasingly downweighted, while others were used more heavily by the optimized prompts. Notably, some down or upweighted sections indeed led to accuracy increases, but not consistently. Equally, Section 1A (“Risk Factors”), which prior work has shown to contain predictive risk sentiment information [ 37 ] , remained largely unchanged. This may indicate that the optimizer failed to infer its predictive usefulness.

#### A.2.5 Agentic.

##### Update-mechanism hyperparameters.

The two update mechanisms plotted in Figure 6 are configured as follows.

ACE (Qwen3-32B). An online batch pipeline runs each batch of 5 5 chains live in the browser, then reflects and curates. The reflector runs once per chain, tagging each existing bullet helpful, harmful, or neutral; the curator runs once per batch over the combined batch reflections with ADD-only operations under a 50,000 50{,}000 -token budget. The reflector/curator LLM is Bedrock qwen.qwen3-32b-v1:0 via the Converse API, with temperature = 0.3 =0.3 and maxTokens = 4096 =4096 . The playbook has five fixed sections ( strategies_and_insights , common_mistakes_to_avoid , problem_solving_heuristics , ui_navigation_tips , others ) and bullets in the format [slug-NNNNN] helpful=H harmful=X :: content . We run 1 1 epoch over 5 5 batches ( ≈ \approx 25–30 chains, no held-out test) and 5 5 seeds ( seed0 – seed4 ), each shuffling the chain order. At evaluation the playbook is prepended to the browser-use system prompt, the agent (qwen3-32b) decodes at factory-default settings, and each cell covers 5 5 seeds × \times 30 30 chains. Per-horizon action budgets and timeouts are L1: 10 10 steps / 300 300 s, L3: 20 20 / 900 900 s, L5: 25 25 / 1200 1200 s, L10: 40 40 / 1800 1800 s.

SFT (Qwen3-8B). We fine-tune Qwen3-8B with LoRA (rank 32 32 , α = 64 \alpha=64 ) on the q,k,v,o,gate,up,down projections, learning rate 5 × 10 − 5 5\times 10^{-5} , gradient accumulation 4 4 , maximum sequence length 16,384 16{,}384 , for 1 1 epoch, since training beyond one epoch overfit and decreased performance on some task-suite tasks. Trajectories are distilled from a teacher agent (Qwen3-32B for L1–L5, Qwen3-235B for L10); per-horizon trajectory and training-example counts are given in Table 10 .

##### Suite construction.

We build the agentic suite on top of WebArena-Infinity [ 61 ] . Chain generation is seeded from two sources per app: the app’s description, which enumerates the available features, and its underlying state files, which fix the concrete entities those features can act on. Conditioning on both lets us generate chains whose steps reference entities that actually exist in the environment. From these we generate sequential chains of length 1 1 , 3 3 , 5 5 , and 10 10 , in which step i + 1 i{+}1 depends on a concrete piece of state that step i i leaves behind in the environment. In Gmail, for instance, the agent creates a label, renames it, and then applies the renamed label to a specific email, so that each later step is only well-defined given the state produced by the earlier ones. The full suite contains 510 510 chains across four apps (Gmail, Gmail Contacts, Handshake, PayPal), spanning the four horizon lengths.

##### Verification.

Each step is paired with a programmatic verifier that reads the app’s state directly rather than parsing the agent’s narration, so success is checked against the environment. A step is marked correct only when its verifier confirms the intended state change, and a chain of length L L succeeds only if all L L of its steps pass. Grounding verification in state ensures that an agent receives no credit for claiming to have completed a step it did not actually perform.

##### Agent harness.

All runs use a browser-use agent driving a headless Chromium against the live app. Within a chain we keep a single agent instance alive across all steps: the browser session is not reset between steps, and each follow-up instruction is appended to the same conversation rather than starting a fresh context, so that when acting on step i + 1 i{+}1 the model sees its own prior trajectory—observations, actions, and errors—from the preceding steps. Each step is given a fixed action budget and is scored independently by its verifier.

##### Zero-shot models.

In addition to the learning methods reported in the main body, we evaluate Deepseek-v3.2 [ 31 ] , GPT-OSS-120B [ 1 ] , and Gemini-2.5-flash-lite [ 7 ] zero-shot under the identical harness. All three degrade sharply as chain length grows (Figure 6 ). The decay tracks the point at which the accumulated trajectory approaches the model’s context window: once earlier trajectory state has to be summarized to fit, the model loses access to the precise environmental cues it needs to ground later actions. Gemini’s larger context window does not rescue this failure mode—it degrades faster than Deepseek despite more available context—suggesting that the bottleneck is the agent’s ability to actually use long trajectory information rather than raw context capacity.

The same “does experience stick across tasks” question, but where the chain arises from the agent’s own actions rather than from a fixed ordering we picked. A learner operates in sequential chains of web tasks built on WebArena Infinity [ 61 ] , where the environment produced by task i i is the starting state of task i + 1 i{+}1 : an action taken early in the chain (an archived email, an applied label, a navigation state) persists and affects every subsequent task’s success. We measure end-to-end pass rate as a function of chain length L ∈ { 1 , 3 , 5 , 10 } L\in\{1,3,5,10\} , across three apps (Gmail, Gmail Accounts, PayPal Wallet) and three backbones (Deepseek-v3.2 [ 31 ] , Gemini-2.5-Flash-Lite [ 7 ] , GPT-OSS-120B [ 1 ] ). This is the closest of our benchmarks to deployed agentic use; we treat it as the natural-regime counterpart to Domain shift’s controlled regime.

## Appendix B Broader Impact

This work develops evaluation infrastructure for continual learning in large language models, with the goal of helping the research community build systems that remain competent as the world changes. Better continual learning methods could have positive societal impact by reducing the need to retrain large models from scratch as new information becomes available, lowering the associated computational cost and energy consumption. At the same time, systems that can efficiently update their knowledge after deployment also carry risks: a model that rapidly incorporates new information could more easily be steered toward harmful or false beliefs through targeted data injection. Our work does not propose a new training method, but to the extent that it accelerates progress in continual learning, these dual-use considerations apply. We do not anticipate direct harms from the framework or evaluation protocols introduced here, as they are designed for measuring and understanding model behavior rather than enabling any specific application.

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .