To Retain or to Adapt? Generalizing Continual Learning — part 5/15

Cleaned-text code points [62800, 77920). Bibliography omitted here; source.md includes it.
The closest theoretical neighbor of our work is the line of analysis on linear-model CL by Evron et al. (2022) ; Goldfarb et al. (2024) . Their analyses bound forgetting for sequential gradient descent on quadratic losses; we share the quadratic regime and the gradient-descent algorithm, but ask a different question — bounding real-time error (ALE) rather than retrospective forgetting — and obtain the Critical Task Duration as the central object of study. Our analysis is most naturally interpreted under the domain-incremental regime, where task identity shifts the input/output distribution while the label space is preserved; this matches both our experimental setting (CLEAR, MD5, Permuted/Shuffled CIFAR-10) and our quadratic-drift theory.

A complementary line of work, recently crystallized as the Big World Hypothesis ( Javed and Sutton, 2024 ) , holds that the world is too large for any bounded learner to model exhaustively—so CL is fundamentally a problem of tracking rather than memorization. This view has technical roots in the early work of Sutton et al. (2007) , and is empirically supported by recent work on the loss of plasticity ( Dohare et al., 2024 ; Dohare et al., 2021 ) . The ALE metric is itself a tracking objective, and our framework can be read as the optimization-theoretic instantiation of this hypothesis.

Within Online Continual Learning ( Cai et al., 2021 ; Lopez-Paz and Ranzato, 2017 ; Buzzega et al., 2020 ) , our Transfer Efficiency decomposition turns the empirical observation of Cai et al. (2021) —that learning efficacy and information retention conflict—into a quantitative trade-off mediated by the Critical Task Duration. Kumar et al. (2025) provide the information-theoretic ALE framing under which we operate; our contribution complements theirs by characterizing which implementable algorithms approach the information-theoretic ceiling and under what conditions. The framework also clarifies the relationship to non-stationary RL theory ( Lecarpentier and Rachelson, 2019 ; Lecarpentier et al., 2021 ; Khetarpal et al., 2022 ) , where similar adaptation–stability trade-offs appear in the context of value-function tracking.

### 7 Discussion and Conclusion

The Continual Learning (CL) literature has long been framed around mitigating catastrophic forgetting. This perspective implicitly assumes that the ideal lifelong learner is the Joint-Task Learning (JTL) solution—a model trained jointly on all data observed so far. In this work, we challenge this assumption, proposing a more general formulation of continual learning that shifts the focus from knowledge retention to real-time adaptation.

Our starting point is the Average Lifelong Error (ALE). Kumar et al. (2025) formalized the trade-offs underlying ALE through an information-theoretic lens, establishing the performance ceiling that a compute-bounded learner can hope to approach. Our framework complements this view by characterizing the optimization-time realization of the same trade-off: which implementable algorithms can approach this ceiling, and under what conditions. We show that real-time performance is governed by the interaction of two stochastic processes—one describing the environment, and the other the learner—and that implementation-level choices (the optimizer, the model, its parametrization, e.g. NTK versus μ \mu P) directly shape the balance between positive and negative transfer.

We compared the JTL agent (which assumes an ergodic and stable past) with the Independent-Task Learning (ITL) agent (which assumes a fully volatile environment) through the lens of Transfer Efficiency ( TE \mathrm{TE} ), a quantity that decomposes into two largely independent components: Instability (the stationary bias from optimizing over conflicting past tasks) and Transient Error (the cost of learning from a naive initialization). Under mild convergence conditions and whenever the average Instability is positive, this decomposition yields a Critical Task Duration ( N max ∞ N_{\max}^{\infty} ): a threshold beyond which the warm-start advantage of JTL is overtaken by the ITL agent’s ability to fit the current environment.

In the quadratic regime, this decomposition admits closed-form expressions that make the tension between stationary bias and the cost of learning fully transparent. Two takeaways carry over to general non-convex settings: (1) the environment’s Growth Rate —the speed at which it expands—is the decisive parameter for the balance between positive and negative transfer; and (2) once the number of tasks is sufficiently large, transfer is primarily determined by the relative weight of the Instability and the ITL Transient, with the JTL Transient acting as a lower-order correction. We empirically verified that these mechanisms persist on continual image-classification benchmarks (CLEAR and MD5) and the reinforcement-learning benchmark MT10.

Recognizing the JTL algorithm as a specific instance of predictive online learning—which rests on the assumption of predictable sequences—suggests a natural generalization: a broader family of Predictive CL algorithms that optimize explicit, environment-specific predictions of the future . As a proof of concept, we introduced the Window Algorithm, which adapts its memory horizon based on the environment’s Mean Squared Displacement (MSD), a local, interval-based analogue of the Growth Rate. In controlled drift settings, this simple approach consistently outperforms both JTL and ITL. More broadly, because the prediction mapping itself can be learned, future algorithms need not commit to a fixed objective a priori, but could instead infer the most appropriate predictive target directly from experience. We believe this represents a promising new direction for continual learning, introducing prediction as an additional axis for designing adaptive learning algorithms.

The broader contribution of this work, however, is conceptual. The framework we develop— Transfer Efficiency, Instability, and Transient Error —together with the Predictive-CL perspective, helps unify continual learning with closely related areas such as online learning, stochastic optimization, multi-task learning, and meta-learning. Beyond introducing new metrics, our framework gives the continual learning community a common language and a unified analytical toolkit for studying real-time adaptation under non-stationary environments.

###### Limitations.

Our conclusions depend on the choice of evaluation objective. ALE captures a natural notion of real-time adaptation under computational constraints, but alternative desiderata such as robustness, coverage, fairness, or worst-case performance may favor different learning strategies and retention mechanisms. Accordingly, we do not claim that ALE is the universally correct objective for continual learning. Rather, our goal is to characterize the behavior and predictions that follow from this particular formulation. Under alternative objectives, the relative performance of JTL, ITL, and other Predictive CL algorithms may differ. The proposed Predictive CL framework remains applicable as a general decomposition and analysis tool, but quantities such as the Critical Task Duration and Transfer Efficiency should be interpreted as properties of the ALE objective rather than universal characteristics of continual learning.

#### Acknowledgements

This research was enabled in part by support provided by (Calcul Québec) (https://www.calculquebec.ca/en/) and the Digital Research Alliance of Canada (https://alliancecan.ca/en). The authors acknowledge the material support of NVIDIA in the form of computational resources. GL would like to acknowledge the support of the AI Center PhD fellowship. Additionally, MS was supported by Fonds de recherche du Québec – Nature et Technologies (FRQNT) #2023-2024-B2X-336394. CV is funded by the Deutsche Forschungsgemeinschaft (DFG) under both the project 468806714 of the Emmy Noether Programme and under Germany’s Excellence Strategy – EXC number 2064/1 – Project number 390727645. CV also thanks the international Max Planck Research School for Intelligent Systems (IMPRS-IS). We acknowledge support by CIFAR (Canada AI Chair; Learning in Machine and Brains Fellowship). We are grateful to David Abel for his generous and insightful feedback on earlier results of this work, which have made this paper better in many ways.

## Appendix

### Appendix A Theoretical proofs

Notation

#### A.1 Introduction to the Framework of Kumar et al. (2025)

###### The Environment.

We define an environment as ℰ = ( A , O , 𝒫 ) \mathcal{E}=(A,O,\mathcal{P}) , where A A and O O denote the action and observation spaces, and 𝒫 ( ⋅ ∣ H t , a t ) \mathcal{P}(\cdot\mid H_{t},a_{t}) is the distribution of the next observation o t + 1 o_{t+1} given the latest action a t a_{t} and the history H t = { ( a 0 , o 1 ) , … , ( a t − 1 , o t ) } H_{t}=\{(a_{0},o_{1}),\dots,(a_{t-1},o_{t})\} . While ℰ \mathcal{E} itself is stationary, the effective observation distribution 𝒫 t  ( o ) = 𝒫 ⁡ ( o ∣ H t , a t ) \mathcal{P}_{t}(o)=\mathcal{P}(o\mid H_{t},a_{t}) is non-stationary due to its dependence on H t H_{t} . Given a reward function r r , the response of the environment to an action-observation pair is the reward r ⁡ ( a t , o t + 1 ) r(a_{t},o_{t+1}) .

###### Agent and Learning Algorithm.

The agent acts according to a stochastic policy π t \pi_{t} sampled from a stochastic process, which we denote as the learning algorithm 𝒜 ( ⋅ ∣ H t ) \mathcal{A}(\cdot\mid H_{t}) 6 6 6 We assume that 𝒜 \mathcal{A} has access to the reward function or the history of collected rewards. In supervised learning, this is equivalent to assuming access to the loss function. . Each history H t H_{t} thus arises sequentially by sampling π t ∼ 𝒜 ( ⋅ ∣ H t ) \pi_{t}\sim\mathcal{A}(\cdot\mid H_{t}) , a t ∼ π t ( ⋅ ∣ H t ) a_{t}\sim\pi_{t}(\cdot\mid H_{t}) , and finally o t + 1 ∼ 𝒫 ( ⋅ ∣ H t , a t ) o_{t+1}\sim\mathcal{P}(\cdot\mid H_{t},a_{t}) .

###### Continual Learning Objective.

Let ℓ ⁡ ( π , o ) \ell(\pi,o) be a non-negative measure of error for a learner policy π \pi evaluated on an observation o o 7 7 7 Note that we bypass explicit action sampling here, as it is incorporated into the definition of ℓ \ell . . We denote by 𝒫 n \mathcal{P}_{n} the conditional distribution 𝒫 ( ⋅ ∣ H n , a n ) \mathcal{P}(\cdot\mid H_{n},a_{n}) . Kumar et al. (2025) define the continual learning objective as the minimization of the real-time error, or Average Lifelong Error (ALE): inf 𝒜 ALE ⁡ ( 𝒜 ) s.t. computational constraints , \inf_{\mathcal{A}}\,\,\mathrm{ALE}(\mathcal{A})\qquad\text{s.t. computational constraints}, (8) where the ALE is defined as: ALE ( 𝒜 ) := 1 T ∑ n = 1 T 𝔼 π n ∼ 𝒜 ( ⋅ ∣ H n ) , o n + 1 ∼ 𝒫 n [ ℓ ( π n , o n + 1 ) ] . \mathrm{ALE}(\mathcal{A}):=\frac{1}{T}\sum_{n=1}^{T}\mathbb{E}_{\pi_{n}\sim\mathcal{A}(\cdot\mid H_{n}),\,o_{n+1}\sim\mathcal{P}_{n}}\left[\ell(\pi_{n},o_{n+1})\right]. (9) Note: Kumar et al. (2025) consider an infinite horizon. While asymptotic averaging is common in reinforcement learning, it can obscure maladaptive behaviors that cause failures over a finite lifespan. In this work, we follow the online learning tradition and focus on the finite-horizon version of the original objective.

###### Supervised Specialization.

In supervised settings, observations are pairs ( x , y ) ∈ X × Y (x,y)\in X\times Y , and actions correspond to predictions y ^ = f ⁡ ( x ) \hat{y}=f(x) . The learning algorithm is conditioned solely on the observation sequence o 1 : t o_{1:t} . For consistency with Kumar et al. (2025) and to maintain general notation, we denote the conditioning on the entire history as H t H_{t} , despite its independence from the agent’s actions in the supervised case. Because 𝒫 n \mathcal{P}_{n} does not depend on the prediction y ^ \hat{y} , the continual learning objective simplifies to: ALE ( 𝒜 ) = 1 T ∑ n = 1 T 𝔼 f ∼ 𝒜 ( ⋅ ∣ H n ) 𝔼 ( x , y ) ∼ 𝒫 n [ ℓ ( f , ( x , y ) ) ] . \mathrm{ALE}(\mathcal{A})=\frac{1}{T}\sum_{n=1}^{T}\mathbb{E}_{f\sim\mathcal{A}(\cdot\mid H_{n})}\,\mathbb{E}_{(x,y)\sim\mathcal{P}_{n}}\,[\ell(f,(x,y))]. (10)

#### A.2 Analysis Setup and Formalism

We consider a sequence of supervised learning tasks where an agent interacts with a piecewise stationary environment, assuming task boundaries are known. We specialize the general lifelong learning framework to the case of stochastic gradient-based optimization.

###### Task-Based Formulation.

We assume the data-generating process is locally stationary , defined formally as follows:

###### Definition (Task-Based Data Generating Process) .

A history H T H_{T} is a task-based sequence if the sampling distribution 𝒫 n \mathcal{P}_{n} is locally stationary. The timeline is partitioned into intervals called tasks S k = [ a k , b k ] S_{k}=[a_{k},b_{k}] . At each time step n ∈ S k n\in S_{k} , an observation o n o_{n} is sampled from a fixed, task-specific distribution 𝒫 k \mathcal{P}^{k} .

###### Batched Observations.

For generality, we assume each observation o t o_{t} is a batch of B ≥ 1 B\geq 1 samples: o t = { ( x b , y b ) } b = 1 B . o_{t}=\{(x_{b},y_{b})\}_{b=1}^{B}. Accordingly, the full history is H t = { o 1 , … , o t } H_{t}=\{o_{1},\dots,o_{t}\} . The model f f operates at the sample level , producing predictions f ⁡ ( x b ) f(x_{b}) for each input in the batch. The batch loss is defined as the average of the per-sample losses.

###### The Data-Generating Process.

Unless stated otherwise, we assume a task-based sequence with known task switch points . We denote the task identity at time n n by z ⁡ ( n ) z(n) and the duration of task k k by N k = | S k | N_{k}=|S_{k}| . At each step n ∈ S k n\in S_{k} , a batch o n o_{n} is sampled from 𝒫 k \mathcal{P}^{k} . The expected loss for task k k over the model parameters θ \theta is: ℒ k  ( θ ) := 𝔼 ( x , y ) ∼ 𝒫 k  [ ℓ ⁡ ( θ , ( x , y ) ) ] . \mathcal{L}^{k}(\theta):=\mathbb{E}_{(x,y)\sim\mathcal{P}^{k}}[\ell(\theta,(x,y))]. (11) The corresponding oracle minimizer is θ ⋆ k = arg ⁡ min θ  ℒ k  ( θ ) \theta^{k}_{\star}=\arg\min_{\theta}\mathcal{L}^{k}(\theta) .

###### Measures of Error and Performance.

Within this formalism, we define the Average Task Error ( ALE k \mathrm{ALE}^{k} ) as the task-specific view of the lifelong ALE: ALE k  ( 𝒜 ) := 1 N k  ∑ n = a k b k 𝔼 θ ∼ Q n  [ ℒ k  ( θ ) ] . \mathrm{ALE}^{k}(\mathcal{A}):=\frac{1}{N_{k}}\sum_{n=a_{k}}^{b_{k}}\mathbb{E}_{\theta\sim Q_{n}}[\mathcal{L}^{k}(\theta)]. (12) It follows that ALE ⁡ ( 𝒜 ) = ∑ k = 1 K w ¯ k ⋅ ALE k  ( 𝒜 ) \mathrm{ALE}(\mathcal{A})=\sum_{k=1}^{K}\bar{w}_{k}\cdot\mathrm{ALE}^{k}(\mathcal{A}) with task weights w ¯ k = N k / T \bar{w}_{k}=N_{k}/T .

The Transfer Efficiency (TE) for task k k quantifies the performance gain of a Joint-Task Learning (JTL) agent over an Independent-Task Learning (ITL) baseline: TE k := ALE k  ( 𝒜 ITL ) − ALE k  ( 𝒜 JTL ) . \mathrm{TE}^{k}:=\mathrm{ALE}^{k}(\mathcal{A}_{\text{ITL}})-\mathrm{ALE}^{k}(\mathcal{A}_{\text{JTL}}). (13)

##### A.2.1 The Algorithm as a Markov Process