To Retain or to Adapt? Generalizing Continual Learning — part 3/15
Cleaned-text code points [31622, 47250). Bibliography omitted here; source.md includes it.
Figure 5 confirms each of the two halves of P1. First, increasing the shift magnitude (PC-16 → \to 32, SC-10 → \to 30 → \to 50) translates the TE curves vertically downward, exactly as the additive − ℐ k -\mathcal{I}^{k} term in Equation 2 predicts. Second, the empirical TE is well fit by a 1 / N 1/N curve across all settings, indicating that the per-step transient gap ℒ k ( Q n ) − ℒ k ( Q ∞ ) \mathcal{L}^{k}(Q_{n})-\mathcal{L}^{k}(Q_{\infty}) vanishes super-linearly — the regime in which Lemma 6 applies. The middle panels additionally confirm an asymptotic prediction we revisit in Section 4 : the JTL Transient is consistently several orders of magnitude smaller than the ITL Transient, so Transfer Efficiency is ultimately driven by Instability and the ITL Transient.
###### (P2) Crossing the Critical Task Duration in CLEAR.
On CLEAR where Instability is small but positive Lemma 6 predicts a finite breakeven horizon. We sweep N ∈ { 3000 , 6000 , 9000 , 12000 } N\in\{3000,6000,9000,12000\} training steps per task and re-measure each component of the decomposition.
Table 2 confirms P2: Transfer Efficiency falls monotonically from + 0.216 +0.216 at N = 3000 N=3000 to + 0.10 +0.10 at N = 12000 N=12000 , while the Instability gap (visible in the closing JTL − - ITL ALE difference) is approximately unchanged. The driver is entirely the ITL Transient δ ITL \delta_{\text{ITL}} , which shrinks as the ITL agent gets more time to converge on each task. Extrapolating the trend places the CLEAR breakeven horizon outside the budget we could run, consistent with CLEAR’s small empirical Instability.
### 4 Quadratic-Loss Analysis
This section makes the transfer mechanisms analytically transparent. We do two things. First, for quadratic losses we derive exact closed-form expressions for the Instability and the Transient Error—and hence for Transfer Efficiency—that expose how each quantity depends on the design choices (step size η \eta , batch size B B , parametrization). Second, we pass to an asymptotic (large- k k ) analysis that collapses these expressions into a single sufficient condition for positive transfer, governed by one environment-level descriptor: the rate at which the environment expands with new tasks. Different environments yield different balances between ITL and JTL, and this descriptor predicts which one wins.
The quadratic setting has been extensively studied in both the stochastic optimization and multi-task learning literatures, allowing us to apply existing results to our specific case. Even in such a simple setting, the interplay of learning dynamics and bias yields multiple layers of complexity, which offer powerful insights into the nature of transfer. The decomposition in Lemmas 5 and 6 is general under task-wise convergence; the closed-form expressions below additionally use standard assumptions detailed in Section A.2 (aligned positive-definite Hessians, small constant step sizes, locally stationary gradient-noise covariances, uniform observation-noise floors). These assumptions are not needed for the definitions themselves, but they make the transfer mechanisms analytically transparent.
#### 4.1 Setup
For every task j ∈ { 1 , … , K } j\in\{1,\dots,K\} , the expected loss is ℒ j ( θ ) = 1 2 ( θ − θ ⋆ j ) ⊤ H j ( θ − θ ⋆ j ) + c j , \mathcal{L}^{j}(\theta)=\tfrac{1}{2}(\theta-\theta_{\star}^{j})^{\top}H_{j}(\theta-\theta_{\star}^{j})+c_{j}, (4) with H j ≻ 0 H_{j}\succ 0 (symmetric positive definite) and c j c_{j} the irreducible loss (Bayes error) for task j j . The Hessian H j H_{j} is data-driven: in linear regression with inputs x ∼ 𝒫 x j x\sim\mathcal{P}^{j}_{x} , it is simply H j = 𝔼 x ∼ 𝒫 x j [ x x ⊤ ] H_{j}=\mathbb{E}_{x\sim\mathcal{P}^{j}_{x}}[xx^{\top}] . The re-centered form above is convenient for the analysis, but the gradient ∇ ℒ j ( θ ) = H j ( θ − θ ⋆ j ) = H j θ − 𝔼 [ x y ] \nabla\mathcal{L}^{j}(\theta)=H_{j}(\theta-\theta_{\star}^{j})=H_{j}\theta-\mathbb{E}[xy] is computable from data without knowing θ ⋆ j \theta_{\star}^{j} ; we use the ( θ ⋆ j , c j ) (\theta_{\star}^{j},c_{j}) parametrization only to expose the dependence of dynamics on the task minimizer.
In this setting, the expected risk for task k k , 𝔼 θ [ ℒ k ( θ n ) ] \mathbb{E}_{\theta}[\mathcal{L}^{k}(\theta_{n})] , is completely determined by the first two moments of the parameter distribution Q n Q_{n} : the mean μ n \mu_{n} and covariance Σ n \Sigma_{n} . Under the gradient-noise assumptions in Section A.2 , the stationary covariance Σ ∞ \Sigma_{\infty} admits a clean decomposition into a transient part and a stationary noise part whose trace scales as 1 / B 1/B in the batch size.
#### 4.2 Closed-form Instability and Transient Error
To express the Instability and Transient Error in closed form, we introduce the relative curvature matrix ℛ k = H dyn , k − 1 H k \mathcal{R}^{k}=H_{\text{dyn},k}^{-1}H_{k} : the curvature of the current task measured against the curvature of the optimization objective (which governs the parameter dynamics). For ITL, ℛ k = I \mathcal{R}^{k}=I is simply the identity. For JTL it generally is not, and the misalignment between dynamics and evaluation is precisely what makes transfer non-trivial.
Throughout this section we use the Mahalanobis-style notation ‖ v ‖ M 2 := v ⊤ M v \|v\|_{M}^{2}:=v^{\top}Mv for any positive-(semi)definite matrix M M . A central quantity is what we call the Task Diversity : the additional variance in the optimization process due to the aggregation of multiple tasks. Recalling the task weights w j k = | S j | / b k w_{j}^{k}=|S_{j}|/b_{k} from Section 2.4 , 𝒱 d i v k = ∑ j = 1 k w j k ⋅ ‖ ∇ ℒ j ( θ ⋆ J T L , k ) ‖ ℛ k 2 , \mathcal{V}_{div}^{k}=\sum_{j=1}^{k}w_{j}^{k}\cdot\left\|\nabla\mathcal{L}^{j}(\theta_{\star}^{JTL,k})\right\|_{\mathcal{R}^{k}}^{2}, where θ ⋆ J T L , k \theta_{\star}^{JTL,k} is the minimizer of the JTL objective at task k k . The norm is taken in the relative-curvature geometry, so tasks whose gradients are large along directions the optimizer treats as informative contribute more.
Within these assumptions, the Instability of task k k admits an exact decomposition into a bias term and a variance term proportional to the Task Diversity (full derivation in Theorem 24 ): ℐ k = ℒ k ( θ ⋆ JTL , k ) − ℒ k ( θ ⋆ k ) ⏟ JTL Bias + η 4 B 𝒱 d i v k ⏟ JTL Variance . \mathcal{I}^{k}=\underbrace{\mathcal{L}^{k}(\theta_{\star}^{\text{JTL},k})-\mathcal{L}^{k}(\theta_{\star}^{k})}_{\text{JTL Bias}}+\,\underbrace{\,\,\frac{\eta}{4\,B}\,\mathcal{V}_{div}^{k}\,\,}_{\text{JTL Variance}}. The two terms describe distinct cost sources of joint training. The JTL Bias is the structural penalty: the JTL solution θ ⋆ JTL , k \theta_{\star}^{\text{JTL},k} is a compromise across the task collection, so when evaluated on a single task it pays a fixed gap relative to that task’s own optimum. The JTL Variance is the dynamical penalty: aggregating tasks injects gradient noise proportional to the step size η \eta and inversely to the batch size B B . This noise is amplifies only along directions that the relative-curvature geometry identifies as informative—namely, directions where task k k ’s curvature is large relative to the pooled objective, corresponding to features that are important for task k k but receive little support from the aggregated gradient of the other tasks.
The Transient Error admits an analogous decomposition ( Theorem 25 ), expressed in terms of the deviations μ ~ n := μ n − μ ∞ \tilde{\mu}_{n}:=\mu_{n}-\mu_{\infty} and Σ ~ n := Σ n − Σ ∞ \tilde{\Sigma}_{n}:=\Sigma_{n}-\Sigma_{\infty} of the mean and covariance from their stationary values: τ δ k = 1 N k ( ‖ μ ~ 0 ‖ ℛ k 2 − ‖ μ ~ N k ‖ ℛ k 2 ) ⏟ Mean Potential Difference + 1 N k ( Tr ( ℛ k Σ ~ 0 ) − Tr ( ℛ k Σ ~ N k ) ) ⏟ Variance Potential Difference , \tau\,\delta^{k}=\frac{1}{N_{k}}\,\underbrace{\left(\|\tilde{\mu}_{0}\|^{2}_{\mathcal{R}^{k}}-\|\tilde{\mu}_{N_{k}}\|^{2}_{\mathcal{R}^{k}}\right)}_{\text{Mean Potential Difference}}+\frac{1}{N_{k}}\,\underbrace{\left(\text{Tr}(\mathcal{R}^{k}\tilde{\Sigma}_{0})-\text{Tr}(\mathcal{R}^{k}\tilde{\Sigma}_{N_{k}})\right)}_{\text{Variance Potential Difference}}, where τ = 4 η \tau=4\eta is the learning time scale. This expression gives a clean physical reading of the cost of learning a task: it is the drop in potential energy between where the agent started the task and where it ended, measured on a quadratic potential surface warped by the relative curvature ℛ k \mathcal{R}^{k} . The transient cost is therefore high precisely when initialization is far from the task’s stationary distribution, and decays as the agent drains that potential.
#### 4.3 Asymptotic Conditions for Positive Transfer
The exact formulas above hold at every k k but are opaque. To expose how transfer depends on the environment, we now take an asymptotic (large- k k ) view, in which the entire ITL/JTL balance is governed by a single environment-level descriptor: the rate at which the environment expands with new tasks. Different environments yield different descriptors, and hence different balances between ITL and JTL. We call this descriptor the Growth Rate .
###### Definition 7 (Growth Rate) .
Let ψ : ℕ → ℝ + \psi:\mathbb{N}\to\mathbb{R}^{+} be a monotonically increasing function such that the expected squared distance of the k k -th task from the origin scales as 𝔼 [ ‖ θ ⋆ k ‖ 2 ] ∈ Θ ( ψ ( k ) ) . \mathbb{E}[\|\theta_{\star}^{k}\|^{2}]\in\Theta(\psi(k)).
The Growth Rate quantifies how fast the environment is expanding. If ψ ( k ) = 1 \psi(k)=1 the environment is bounded , meaning a finite diameter M M contains all task-solution vectors. If ψ ( k ) = k 2 \psi(k)=k^{2} the environment has a linear drift : tasks move with constant velocity ∂ k θ ⋆ k = v \partial_{k}\theta_{\star}^{k}=v . Between these extremes, a diffusive environment ( ψ ( k ) = k \psi(k)=k ) corresponds to tasks drifting via a random walk. Figure 6 visualizes these three cases.
We can characterize the rate at which each component of the Transfer Efficiency scales, as a function of the Growth Rate. Putting everything together we write a sufficient asymptotic condition for positive transfer:
###### Theorem 8 (Asymptotic behaviour of Transfer for Quadratic Objectives — Informal) .
In the setup described above, in an environment with Growth Rate Θ ( ψ ( k ) ) \Theta(\psi(k)) we have ℐ k ∈ Θ ( ψ ( k ) ) , δ I T L k ∈ Θ ( ψ ( k ) N k ) , δ J T L k ∈ Θ ( ψ ( k ) k 2 + ψ ′ ( k ) ) , \mathcal{I}^{k}\in\Theta(\psi(k)),\quad\delta^{k}_{ITL}\in\Theta\left(\tfrac{\psi(k)}{N_{k}}\right),\quad\delta^{k}_{JTL}\in\Theta\left(\tfrac{\psi(k)}{k^{2}}+\psi^{\prime}(k)\right), where ψ ′ ( k ) \psi^{\prime}(k) denotes the environment acceleration . Consequently, for environments with polynomial growth rates the expected Transfer Efficiency is positive for large k k if ℐ k < C I T L k 𝔼 [ ‖ μ ~ 0 I T L , k ‖ 2 ] N k , \mathcal{I}^{k}<C^{k}_{ITL}\,\frac{\mathbb{E}[\|\tilde{\mu}_{0}^{ITL,k}\|^{2}]}{N_{k}}, where C I T L k C^{k}_{ITL} is a constant of the bound.
An extended version of this result and its proof can be found in Section A.5 .
The three rates have a clear interpretation. The Instability inherits the full ψ ( k ) \psi(k) scaling: a faster-expanding environment makes the JTL stationary compromise progressively worse on each new task. The ITL Transient is the full ψ ( k ) \psi(k) paid back through N k N_{k} steps: when each task is long, the comparative cost of restarting learning from scratch is low. The JTL Transient, by contrast, is dominated by the much smaller terms ψ ( k ) / k 2 \psi(k)/k^{2} and ψ ′ ( k ) \psi^{\prime}(k) : as k k grows, the JTL agent only has to track how the environment is moving , not where it currently is , which is comparatively cheap.
For polynomial growth, Transfer Efficiency is therefore asymptotically dominated by two competing terms: the Instability ℐ k \mathcal{I}^{k} (the JTL stationary-bias penalty) and the ITL Transient δ I T L k \delta^{k}_{ITL} (the ITL reset penalty); the JTL Transient is lower-order in the regimes we analyze. The sufficient condition in Theorem 8 reads exactly this trade-off: transfer is positive when the JTL stationary penalty is paid back within the budget the ITL agent needs to warm up. From this we rediscover the conventional wisdom that joint-task training is most valuable in the “ few-shot ” regime — where N k N_{k} is small and ITL has no time to recover from a cold start — and ceases to help when the environment is so unpredictable that the bias cost dominates.
#### 4.4 The Continuous Drift Model
To make the asymptotic story concrete, we instantiate the framework on a diffusive regime : a Wiener-walk model where optimal parameters evolve as θ ⋆ k = θ ⋆ k − 1 + Δ k \theta_{\star}^{k}=\theta_{\star}^{k-1}+\Delta_{k} with Δ k ∼ 𝒩 ( 0 , Σ Δ ) \Delta_{k}\sim\mathcal{N}(0,\Sigma_{\Delta}) , and drift magnitude σ Δ 2 := Tr ( Σ Δ ) \sigma_{\Delta}^{2}:=\text{Tr}(\Sigma_{\Delta}) controlling both how far the process may travel and how fast the relevance of past data decays. This regime captures real-world continuous distribution shifts such as the slow evolution of user preferences or weather variation in autonomous navigation. The full setup (Gaussian inputs with covariance Σ x \Sigma_{x} , label noise σ ξ 2 \sigma_{\xi}^{2} ) is in Section A.6 .
Applying the exact Instability and Transient Error formulas to this setup yields a non-asymptotic sufficient condition for positive transfer : a single inequality balancing an ITL Transient cost (which grows linearly in k k but decays as 1 / N 1/N ), a JTL Instability cost (which grows linearly in k k with no compensating 1 / N 1/N factor), and a smaller JTL Transient cost (which decays in both k k and N N ). The three terms mirror Theorem 8 ; the full expression is given in Equation 94 .
Simulations of this model ( Figure 7 ; details in Section B.1 ) confirm the boundary at finite k k and show the positive-transfer region shrinking as k k grows, since the JTL Transient is lower-order than the linear-in- k k terms. In the limit k → ∞ k\to\infty , the condition reduces to a clean closed form for the Critical Task Duration: N max ∞ = 3 γ min σ Δ 2 2 η Tr ( Σ x Σ Δ ) . N_{\max}^{\infty}=\frac{3\gamma_{\min}\,\sigma^{2}_{\Delta}}{2\eta\,\text{Tr}(\Sigma_{x}\Sigma_{\Delta})}. (5) This asymptotic boundary is set by the geometry of the environment together with the optimizer : it depends on the input–drift coupling Tr ( Σ x Σ Δ ) \text{Tr}(\Sigma_{x}\Sigma_{\Delta}) , the loss curvature γ min \gamma_{\min} , and the learning rate η \eta (smaller η \eta favours biased initializations such as JTL), while being invariant to the overall drift magnitude σ Δ 2 \sigma^{2}_{\Delta} and independent of the batch size B B . The full asymptotic landscape and a trajectory-level view of drift eroding the warm-start advantage are in Figures 10 and 25 .
#### 4.5 The Optimization Bias Matters
The closed-form analysis above takes the function class as given. A complementary question is how much of the transfer profile is set by the function class versus the optimization bias of the algorithm that fits it. We isolate this by extending the Continuous Drift Model to a deep-linear teacher-student setup (following Saxe et al. 2014 ; details in Section B.1 ) and comparing two parametrizations of the same linear function class: NTP ( lazy regime , Jacot et al. 2018 ) and μ \mu P ( rich regime ).