To Retain or to Adapt? Generalizing Continual Learning — part 10/15

Cleaned-text code points [141123, 154774). Bibliography omitted here; source.md includes it.

• 𝔼 ⁡ [ | Tr ​ ( Σ ~ 0 JTL , k ) | ] \mathbb{E}[|\text{Tr}(\tilde{\Sigma}_{0}^{\text{JTL},k})|] : Similarly, by Lemma 28 and Lemma 31 , the covariance initialization error is dominated by the covariance drift Θ ​ ( ψ ′ ​ ( k ) ) \Theta(\psi^{\prime}(k)) .

Substituting these scalings back into the inequality for sufficiently large k k : τ ​ N ⋅ Θ ⁡ ( ψ ⁡ ( k ) ) < C ITL k ​ ( Θ ⁡ ( ψ ⁡ ( k ) ) + Θ ⁡ ( 1 ) ) − C JTL k ​ ( Θ ⁡ ( ψ ⁡ ( k ) k 2 ) + Θ ⁡ ( ψ ′ ​ ( k ) ) ) . \tau N\cdot\Theta(\psi(k))<C^{k}_{\text{ITL}}\left(\Theta(\psi(k))+\Theta(1)\right)-C^{k}_{\text{JTL}}\left(\Theta\left(\frac{\psi(k)}{k^{2}}\right)+\Theta(\psi^{\prime}(k))\right). The JTL transient term (the negative contribution) becomes asymptotically negligible compared to the ITL transient term (the positive contribution) because ψ ⁡ ( k ) k 2 ≪ ψ ⁡ ( k ) \frac{\psi(k)}{k^{2}}\ll\psi(k) and ψ ′ ​ ( k ) ≪ ψ ⁡ ( k ) \psi^{\prime}(k)\ll\psi(k) for any polynomial growth.

Therefore, the asymptotic condition simplifies to a direct competition between the ITL transient energy and the JTL instability: τ ​ N ⋅ 𝔼 ⁡ [ ℐ k ] < C ITL k ⋅ 𝔼 ⁡ [ ‖ μ ~ 0 ITL ‖ 2 ] . \tau N\cdot\mathbb{E}[\mathcal{I}^{k}]<C^{k}_{\text{ITL}}\cdot\mathbb{E}[\|\tilde{\mu}_{0}^{\text{ITL}}\|^{2}]. Rearranging for N N , positive expected transfer occurs only if the task duration is sufficiently short relative to the instability: N < C ITL k τ ​ 𝔼 ⁡ [ ‖ μ ~ 0 ITL ‖ 2 ] 𝔼 ⁡ [ ℐ k ] . N<\frac{C^{k}_{\text{ITL}}}{\tau}\frac{\mathbb{E}[\|\tilde{\mu}_{0}^{\text{ITL}}\|^{2}]}{\mathbb{E}[\mathcal{I}^{k}]}. This concludes the proof: expected transfer is positive strictly in the "transient regime", where the benefit of initializing near the task centroid (avoiding re-learning) outweighs the cost of the asymptotic bias and task conflict. ∎

#### A.6 (Non-asymptotic) Continuous Drift Analysis in the Quadratic Model

We now instantiate the general Transfer Efficiency framework for the specific case of drifting linear tasks. This allows us to derive precise, non-asymptotic conditions for the stability-plasticity phase transition observed in our experiments.

##### A.6.1 Setup and Definitions

We assume the environment consists of a sequence of tasks k = 1 , … , K k=1,\dots,K governed by a random walk process. The data inputs x ∈ ℝ d x\in\mathbb{R}^{d} are drawn from a fixed distribution with covariance matrix Σ x ≻ 0 \Sigma_{x}\succ 0 . The target for task k k is given by y = ( θ ⋆ k ) ⊤ ​ x + ξ y=(\theta_{\star}^{k})^{\top}x+\xi , with label noise ξ ∼ 𝒩 ⁡ ( 0 , σ ξ 2 ) \xi\sim\mathcal{N}(0,\sigma_{\xi}^{2}) . The expected loss function for task k k (over the data distribution) is: ℒ k ​ ( θ ) = 1 2 ​ ‖ θ − θ ⋆ k ‖ Σ x 2 + 1 2 ​ σ ξ 2 , \mathcal{L}^{k}(\theta)=\frac{1}{2}\|\theta-\theta_{\star}^{k}\|_{\Sigma_{x}}^{2}+\frac{1}{2}\sigma_{\xi}^{2}, (77) using the Mahalanobis-style norm ‖ v ‖ M 2 := v ⊤ ​ M ​ v \|v\|_{M}^{2}:=v^{\top}Mv introduced in the main text ( Section 4 ). This expression is the instantiation of the quadratic loss of Section 4 with H j = Σ x H_{j}=\Sigma_{x} (the same data Hessian shared across tasks) and the irreducible loss c j = 1 2 ​ σ ξ 2 c_{j}=\tfrac{1}{2}\sigma_{\xi}^{2} . The optimal parameters evolve according to a Wiener process across tasks: θ ⋆ k = θ ⋆ k − 1 + Δ k , Δ k ∼ 𝒩 ⁡ ( 0 , Σ Δ ) , \theta_{\star}^{k}=\theta_{\star}^{k-1}+\Delta_{k},\quad\Delta_{k}\sim\mathcal{N}(0,\Sigma_{\Delta}), (78) where Σ Δ \Sigma_{\Delta} is the drift covariance matrix. Both the Independent-Task Learning (ITL) and Joint-Task Learning (JTL) agents optimize these objectives using SGD with a constant step-size η \eta . For simplicity, we assume all tasks have equal duration N N .

As we will demonstrate in the derivations below, this type of non-stationarity exhibits a Diffusive growth rate ( ψ ⁡ ( k ) = k \psi(k)=k ), consistent with the regimes defined in Table 4 . Unless otherwise noted, expectations 𝔼 ⁡ [ ⋅ ] \mathbb{E}[\cdot] in this section are taken over the random walk process defining the task sequence.

##### A.6.2 Derivation of Expected Instability

The JTL agent optimizes the uniform mixture of all observed tasks, converging to the cumulative centroid θ ⋆ JTL , k = 1 k ​ ∑ j = 1 k θ ⋆ j \theta_{\star}^{\text{JTL},k}=\frac{1}{k}\sum_{j=1}^{k}\theta_{\star}^{j} . We apply the Exact Instability Decomposition from Theorem 24 to task k k . Since the Hessian H k = Σ x H_{k}=\Sigma_{x} is constant across tasks, the instability for a given realization decomposes into: ℐ k ≈ 1 2 ​ ‖ θ ⋆ JTL , k − θ ⋆ k ‖ Σ x 2 + η 4 ​ k ​ B ​ ∑ j = 1 k ‖ θ ⋆ JTL , k − θ ⋆ j ‖ Σ x 2 2 . \mathcal{I}^{k}\approx\frac{1}{2}\|\theta_{\star}^{\text{JTL},k}-\theta_{\star}^{k}\|_{\Sigma_{x}}^{2}+\frac{\eta}{4kB}\sum_{j=1}^{k}\|\theta_{\star}^{\text{JTL},k}-\theta_{\star}^{j}\|_{\Sigma_{x}^{2}}^{2}. (79)

We first express the lag vector v k := θ ⋆ JTL , k − θ ⋆ k v_{k}:=\theta_{\star}^{\text{JTL},k}-\theta_{\star}^{k} in terms of the sequence of drift increments Δ m \Delta_{m} . Using the backward recursion θ ⋆ j = θ ⋆ k − ∑ m = j + 1 k Δ m \theta_{\star}^{j}=\theta_{\star}^{k}-\sum_{m=j+1}^{k}\Delta_{m} , we substitute into the centroid definition: v k = ( 1 k ∑ j = 1 k θ ⋆ j ) − θ ⋆ k = 1 k ∑ j = 1 k ( θ ⋆ k − ∑ m = j + 1 k Δ m ) − θ ⋆ k = − 1 k ∑ j = 1 k ∑ m = j + 1 k Δ m . v_{k}=\left(\frac{1}{k}\sum_{j=1}^{k}\theta_{\star}^{j}\right)-\theta_{\star}^{k}=\frac{1}{k}\sum_{j=1}^{k}\left(\theta_{\star}^{k}-\sum_{m=j+1}^{k}\Delta_{m}\right)-\theta_{\star}^{k}=-\frac{1}{k}\sum_{j=1}^{k}\sum_{m=j+1}^{k}\Delta_{m}. (80) Rearranging the double summation to group terms by the drift increment Δ m \Delta_{m} (noting that a specific Δ m \Delta_{m} appears whenever j < m j<m , meaning it appears m − 1 m-1 times): v k = − 1 k ∑ m = 2 k ( m − 1 ) Δ m . v_{k}=-\frac{1}{k}\sum_{m=2}^{k}(m-1)\Delta_{m}. (81)

The second term in the instability equation involves the sum of squared deviations from the centroid (the task scatter). This can be rewritten using pairwise squared differences: ∑ j = 1 k ( θ ⋆ JTL , k − θ ⋆ j ) ​ ( θ ⋆ JTL , k − θ ⋆ j ) ⊤ = 1 2 ​ k ​ ∑ i = 1 k ∑ j = 1 k ( θ ⋆ i − θ ⋆ j ) ​ ( θ ⋆ i − θ ⋆ j ) ⊤ . \sum_{j=1}^{k}(\theta_{\star}^{\text{JTL},k}-\theta_{\star}^{j})(\theta_{\star}^{\text{JTL},k}-\theta_{\star}^{j})^{\top}=\frac{1}{2k}\sum_{i=1}^{k}\sum_{j=1}^{k}(\theta_{\star}^{i}-\theta_{\star}^{j})(\theta_{\star}^{i}-\theta_{\star}^{j})^{\top}. (82) The difference between any two optima is simply the sum of the intermediate drift steps: θ ⋆ i − θ ⋆ j = ∑ m = j + 1 i Δ m \theta_{\star}^{i}-\theta_{\star}^{j}=\sum_{m=j+1}^{i}\Delta_{m} (for i > j i>j ).

We now compute the expected instability 𝔼 ⁡ [ ℐ k ] \mathbb{E}[\mathcal{I}^{k}] over the distribution of random walks. The covariance of the lag vector v k v_{k} is the weighted sum of the increment covariances: 𝔼 ⁡ [ v k ​ v k ⊤ ] = 1 k 2 ​ ∑ m = 2 k ( m − 1 ) 2 ​ Σ Δ = 1 k 2 ​ ( ( k − 1 ) ​ k ​ ( 2 ​ k − 1 ) 6 ) ​ Σ Δ . \mathbb{E}[v_{k}v_{k}^{\top}]=\frac{1}{k^{2}}\sum_{m=2}^{k}(m-1)^{2}\Sigma_{\Delta}=\frac{1}{k^{2}}\left(\frac{(k-1)k(2k-1)}{6}\right)\Sigma_{\Delta}. (83) Using the continuous approximation ∑ p = 1 k p 2 ≈ k 3 / 3 \sum_{p=1}^{k}p^{2}\approx k^{3}/3 for large k k : 𝔼 ⁡ [ 1 2 ​ ‖ v k ‖ Σ x 2 ] ≈ 1 2 ​ Tr ​ ( Σ x ⋅ k 3 ​ Σ Δ ) = k 6 ​ Tr ​ ( Σ x ​ Σ Δ ) . \mathbb{E}\left[\frac{1}{2}\|v_{k}\|_{\Sigma_{x}}^{2}\right]\approx\frac{1}{2}\text{Tr}\left(\Sigma_{x}\cdot\frac{k}{3}\Sigma_{\Delta}\right)=\frac{k}{6}\text{Tr}(\Sigma_{x}\Sigma_{\Delta}). (84)

For the task scatter, since 𝔼 ⁡ [ ( θ ⋆ i − θ ⋆ j ) ​ ( θ ⋆ i − θ ⋆ j ) ⊤ ] = | i − j | ​ Σ Δ \mathbb{E}[(\theta_{\star}^{i}-\theta_{\star}^{j})(\theta_{\star}^{i}-\theta_{\star}^{j})^{\top}]=|i-j|\Sigma_{\Delta} , the expected sum of pairwise distances is: 𝔼 ⁡ [ Scatter ] = 1 2 ​ k ​ ∑ i , j | i − j | ​ Σ Δ ≈ 1 2 ​ k ​ ( k 3 3 ) ​ Σ Δ = k 2 6 ​ Σ Δ . \mathbb{E}[\text{Scatter}]=\frac{1}{2k}\sum_{i,j}|i-j|\Sigma_{\Delta}\approx\frac{1}{2k}\left(\frac{k^{3}}{3}\right)\Sigma_{\Delta}=\frac{k^{2}}{6}\Sigma_{\Delta}. (85) Substituting this into the variance trace term: 𝔼 ⁡ [ η 4 ​ k ​ B ​ ∑ j = 1 k ‖ θ ⋆ JTL , k − θ ⋆ j ‖ Σ x 2 2 ] = η 4 ​ k ​ B ​ Tr ​ ( Σ x 2 ⋅ k 2 6 ​ Σ Δ ) = η ​ k 24 ​ B ​ Tr ​ ( Σ x 2 ​ Σ Δ ) . \mathbb{E}\left[\frac{\eta}{4kB}\sum_{j=1}^{k}\|\theta_{\star}^{\text{JTL},k}-\theta_{\star}^{j}\|_{\Sigma_{x}^{2}}^{2}\right]=\frac{\eta}{4kB}\text{Tr}\left(\Sigma_{x}^{2}\cdot\frac{k^{2}}{6}\Sigma_{\Delta}\right)=\frac{\eta k}{24B}\text{Tr}(\Sigma_{x}^{2}\Sigma_{\Delta}). (86)

Combining the expected bias and variance components, the final expected instability is: 𝔼 ⁡ [ ℐ k ] ≈ k 6 ​ Tr ​ ( Σ x ​ Σ Δ ) + η ​ k 24 ​ B ​ Tr ​ ( Σ x 2 ​ Σ Δ ) . \mathbb{E}[\mathcal{I}^{k}]\approx\frac{k}{6}\text{Tr}(\Sigma_{x}\Sigma_{\Delta})+\frac{\eta k}{24B}\text{Tr}(\Sigma_{x}^{2}\Sigma_{\Delta}). (87)

##### A.6.3 Derivation of Expected Transient Error

The Transient Error δ k \delta^{k} is proportional to the energy of the initial parameter error relative to the current task’s stationary distribution. For a specific realization, this is approximated by: δ k ≈ 1 N ​ τ ​ Tr ​ ( ξ N ​ ( Σ x ) ​ ( μ ~ 0 ​ μ ~ 0 ⊤ + Σ ~ 0 ) ) . \delta^{k}\approx\frac{1}{N\tau}\text{Tr}\left(\xi_{N}(\Sigma_{x})\left(\tilde{\mu}_{0}\tilde{\mu}_{0}^{\top}+\tilde{\Sigma}_{0}\right)\right). (88)

###### ITL Agent.

The ITL agent resets to a fixed prior θ 0 ∼ Q 0 \theta_{0}\sim Q_{0} with covariance Σ 0 ITL \Sigma_{0}^{\text{ITL}} . The initial error vector is μ ~ 0 ITL = θ 0 − θ ⋆ k \tilde{\mu}_{0}^{\text{ITL}}=\theta_{0}-\theta_{\star}^{k} . The expected energy of this error depends entirely on how far the random walk has drifted from the origin: 𝔼 ⁡ [ ‖ θ 0 − θ ⋆ k ‖ 2 ] = 𝔼 ⁡ [ ‖ θ 0 ‖ 2 ] + Tr ​ ( ∑ j = 1 k 𝔼 ⁡ [ Δ j ​ Δ j ⊤ ] ) = Tr ​ ( Σ 0 ITL ) + k ​ Tr ​ ( Σ Δ ) . \mathbb{E}[\|\theta_{0}-\theta_{\star}^{k}\|^{2}]=\mathbb{E}[\|\theta_{0}\|^{2}]+\text{Tr}\left(\sum_{j=1}^{k}\mathbb{E}[\Delta_{j}\Delta_{j}^{\top}]\right)=\text{Tr}\left(\Sigma_{0}^{\text{ITL}}\right)+k\text{Tr}(\Sigma_{\Delta}). (89) Let γ min := λ min ​ ( ξ N ​ ( Σ x ) ) \gamma_{\min}:=\lambda_{\min}(\xi_{N}(\Sigma_{x})) . Because Σ ~ 0 ITL = Σ 0 ITL − Σ ∞ ITL \tilde{\Sigma}_{0}^{\text{ITL}}=\Sigma_{0}^{\text{ITL}}-\Sigma_{\infty}^{\text{ITL}} , the expected ITL Transient Error is bounded by: 𝔼 ⁡ [ δ ITL k ] \displaystyle\mathbb{E}[\delta^{k}_{\text{ITL}}] ≥ γ min τ ​ N ​ ( 𝔼 ⁡ [ ‖ μ ~ 0 ITL ‖ 2 ] + Tr ​ ( Σ ~ 0 ITL ) ) \displaystyle\geq\frac{\gamma_{\min}}{\tau N}\left(\mathbb{E}[\|\tilde{\mu}_{0}^{\text{ITL}}\|^{2}]+\text{Tr}(\tilde{\Sigma}_{0}^{\text{ITL}})\right) = γ min τ ​ N ​ ( k ​ Tr ​ ( Σ Δ ) + 2 ​ Tr ​ ( Σ 0 ITL ) − Tr ​ ( Σ ∞ ITL ) ) . \displaystyle=\frac{\gamma_{\min}}{\tau N}\left(k\text{Tr}(\Sigma_{\Delta})+2\text{Tr}\left(\Sigma_{0}^{\text{ITL}}\right)-\text{Tr}({\Sigma}^{\text{ITL}}_{\infty})\right). (90) By Lemma 20 , Tr ​ ( Σ ∞ ITL ) = η ​ σ ξ 2 2 ​ B \text{Tr}({\Sigma}^{\text{ITL}}_{\infty})=\frac{\eta\sigma_{\xi}^{2}}{2B} . The trace of the prior Tr ​ ( Σ 0 ITL ) \text{Tr}\left(\Sigma_{0}^{\text{ITL}}\right) is a constant, which we denote by σ 0 2 \sigma_{0}^{2} .

###### JTL Agent.

The JTL agent initializes at the previous task’s converged centroid θ ⋆ JTL , k − 1 \theta_{\star}^{\text{JTL},k-1} . The expected initialization error decomposes into the geometric drift and the residual contraction: μ ~ 0 JTL , k = Δ ​ μ k + r μ k − 1 . \tilde{\mu}_{0}^{\text{JTL},k}=\Delta\mu_{k}+r_{\mu}^{k-1}. First, we relate the geometric drift Δ ​ μ k \Delta\mu_{k} to the lag vector v k v_{k} . Recall θ ⋆ JTL , k = 1 k ​ θ ⋆ k + k − 1 k ​ θ ⋆ JTL , k − 1 \theta_{\star}^{\text{JTL},k}=\frac{1}{k}\theta_{\star}^{k}+\frac{k-1}{k}\theta_{\star}^{\text{JTL},k-1} . Rearranging for the increment: Δ ​ μ k = θ ⋆ JTL , k − θ ⋆ JTL , k − 1 = θ ⋆ JTL , k − k k − 1 ​ ( θ ⋆ JTL , k − 1 k ​ θ ⋆ k ) = 1 k − 1 ​ ( θ ⋆ k − θ ⋆ JTL , k ) = − 1 k − 1 ​ v k . \Delta\mu_{k}=\theta_{\star}^{\text{JTL},k}-\theta_{\star}^{\text{JTL},k-1}=\theta_{\star}^{\text{JTL},k}-\frac{k}{k-1}\left(\theta_{\star}^{\text{JTL},k}-\frac{1}{k}\theta_{\star}^{k}\right)=\frac{1}{k-1}(\theta_{\star}^{k}-\theta_{\star}^{\text{JTL},k})=-\frac{1}{k-1}v_{k}. The expected squared norm of the geometric drift is therefore: 𝔼 ⁡ [ ‖ Δ ​ μ k ‖ 2 ] = 1 ( k − 1 ) 2 ​ Tr ​ ( 𝔼 ⁡ [ v k ​ v k ⊤ ] ) = 2 ​ k − 1 6 ​ k ​ ( k − 1 ) ​ Tr ​ ( Σ Δ ) . \mathbb{E}[\|\Delta\mu_{k}\|^{2}]=\frac{1}{(k-1)^{2}}\text{Tr}(\mathbb{E}[v_{k}v_{k}^{\top}])=\frac{2k-1}{6k(k-1)}\text{Tr}(\Sigma_{\Delta}). (91) Let d k := 𝔼 ⁡ [ ‖ Δ ​ μ k ‖ 2 ] d_{k}:=\sqrt{\mathbb{E}[\|\Delta\mu_{k}\|^{2}]} be the RMS drift intensity. For k ≥ 2 k\geq 2 , define the RMS initialization error e k := 𝔼 ⁡ [ ‖ μ ~ 0 JTL , k ‖ 2 ] e_{k}:=\sqrt{\mathbb{E}[\|\tilde{\mu}_{0}^{\text{JTL},k}\|^{2}]} . By Minkowski’s inequality, it satisfies the stable recursion e k ≤ d k + ρ μ ​ e k − 1 e_{k}\leq d_{k}+\rho_{\mu}e_{k-1} . Because the drift d k d_{k} changes slowly compared to the exponential decay rate ρ μ \rho_{\mu} (the quasi-static approximation), the error tracks the input scaled by the effective time constant 1 1 − ρ μ \frac{1}{1-\rho_{\mu}} . For k ≫ 1 k\gg 1 , we approximate d k ≈ 1 3 ​ k ​ Tr ​ ( Σ Δ ) d_{k}\approx\sqrt{\frac{1}{3k}\text{Tr}(\Sigma_{\Delta})} . This yields the explicit expected transient energy curve: 𝔼 ⁡ [ ‖ μ ~ 0 JTL , k ‖ 2 ] ≈ ( 1 1 − ρ μ ) 2 ​ Tr ​ ( Σ Δ ) 3 ​ k . \mathbb{E}[\|\tilde{\mu}_{0}^{\text{JTL},k}\|^{2}]\approx\left(\frac{1}{1-\rho_{\mu}}\right)^{2}\frac{\text{Tr}(\Sigma_{\Delta})}{3k}. (92) This confirms that the JTL expected initialization error vanishes as 1 / k 1/k , but its magnitude is inflated by the conditioning of the optimization problem via the ( 1 − ρ μ ) − 2 (1-\rho_{\mu})^{-2} factor.