To Retain or to Adapt? Generalizing Continual Learning — part 7/15

Cleaned-text code points [93765, 109369). Bibliography omitted here; source.md includes it.

The SGD update is θ n + 1 = θ n − η ​ g ¯ n \theta_{n+1}=\theta_{n}-\eta\bar{g}_{n} . Let μ n = 𝔼 ⁡ [ θ n ] \mu_{n}=\mathbb{E}[\theta_{n}] be the mean parameter and θ ~ n = θ n − μ n \tilde{\theta}_{n}=\theta_{n}-\mu_{n} be the centered iterate. The mean evolves as μ n + 1 = μ n − η ​ 𝔼 ​ [ g ¯ n ] \mu_{n+1}=\mu_{n}-\eta\mathbb{E}[\bar{g}_{n}] . For quadratic losses, 𝔼 ⁡ [ g ¯ n ] = H ⁡ ( μ n − θ ⋆ ) \mathbb{E}[\bar{g}_{n}]=H(\mu_{n}-\theta_{\star}) . The evolution of the fluctuation is: θ ~ n + 1 = θ n + 1 − μ n + 1 = θ ~ n − η ⁡ ( g ¯ n − 𝔼 ⁡ [ g ¯ n ] ) . \tilde{\theta}_{n+1}=\theta_{n+1}-\mu_{n+1}=\tilde{\theta}_{n}-\eta(\bar{g}_{n}-\mathbb{E}[\bar{g}_{n}]). Expanding g ¯ n \bar{g}_{n} around the mean μ n \mu_{n} : g ¯ n ≈ H ⁡ ( θ n − θ ⋆ ) + ξ n = H ​ θ ~ n + H ⁡ ( μ n − θ ⋆ ) + ξ n , \bar{g}_{n}\approx H(\theta_{n}-\theta_{\star})+\xi_{n}=H\tilde{\theta}_{n}+H(\mu_{n}-\theta_{\star})+\xi_{n}, where ξ n \xi_{n} is the zero-mean batch noise with covariance C n C_{n} . Substituting this into the fluctuation update: θ ~ n + 1 = ( I − η ​ H ) ​ θ ~ n − η ​ ξ n . \tilde{\theta}_{n+1}=(I-\eta H)\tilde{\theta}_{n}-\eta\xi_{n}. The covariance Σ n + 1 = 𝔼 ⁡ [ θ ~ n + 1 ​ θ ~ n + 1 ⊤ ] \Sigma_{n+1}=\mathbb{E}[\tilde{\theta}_{n+1}\tilde{\theta}_{n+1}^{\top}] is then: Σ n + 1 = ( I − η ​ H ) ​ 𝔼 ​ [ θ ~ n ​ θ ~ n ⊤ ] ​ ( I − η ​ H ) ⊤ + η 2 ​ 𝔼 ​ [ ξ n ​ ξ n ⊤ ] , \Sigma_{n+1}=(I-\eta H)\mathbb{E}[\tilde{\theta}_{n}\tilde{\theta}_{n}^{\top}](I-\eta H)^{\top}+\eta^{2}\mathbb{E}[\xi_{n}\xi_{n}^{\top}], where the cross-terms vanish because ξ n \xi_{n} is independent of θ ~ n \tilde{\theta}_{n} . Substituting C n = 𝔼 ⁡ [ ξ n ​ ξ n ⊤ ] C_{n}=\mathbb{E}[\xi_{n}\xi_{n}^{\top}] yields the first result.

Solving for stationarity ( Σ n + 1 = Σ n = Σ ∞ \Sigma_{n+1}=\Sigma_{n}=\Sigma_{\infty} ) gives: Σ ∞ = ( I − η ​ H ) ​ Σ ∞ ​ ( I − η ​ H ) ⊤ + η 2 ​ C ∞ . \Sigma_{\infty}=(I-\eta H)\Sigma_{\infty}(I-\eta H)^{\top}+\eta^{2}C_{\infty}. Expanding and ignoring the higher-order η 2 ​ H ​ Σ ∞ ​ H ⊤ \eta^{2}H\Sigma_{\infty}H^{\top} term (first-order continuous-time approximation) yields the Lyapunov equation η ​ H ​ Σ ∞ + η ​ Σ ∞ ​ H = η 2 ​ C ∞ \eta H\Sigma_{\infty}+\eta\Sigma_{\infty}H=\eta^{2}C_{\infty} . ∎

###### Lemma 20 (ITL Gradient Noise and Parameter Covariance) .

Consider a learning process with batch size B ≥ 1 B\geq 1 and constant step-size η \eta . If the target noise is i.i.d. sampled from a zero-mean Gaussian ϵ ∼ 𝒩 ⁡ ( 0 , σ 2 ) \epsilon\sim\mathcal{N}(0,\sigma^{2}) , the gradient noise covariance for a single task is locally constant along the optimization trajectory, taking the form: C n ≈ C ∞ = σ 2 B ​ H . C_{n}\approx C_{\infty}=\frac{\sigma^{2}}{B}H. Consequently, by Lemma 19 , for small step-sizes, the stationary parameter covariance matrix Σ ∞ \Sigma_{\infty} is isotropic: Σ ∞ = η ​ σ 2 2 ​ B ​ I . \Sigma_{\infty}=\frac{\eta\sigma^{2}}{2B}I.

###### Proof.

Consider the quadratic loss ℓ ⁡ ( θ , ( x , y ) ) = 1 2 ​ ( y − θ ⊤ ​ x ) 2 \ell(\theta,(x,y))=\frac{1}{2}(y-\theta^{\top}x)^{2} where y = θ ⋆ ⊤ ​ x + ϵ y=\theta_{\star}^{\top}x+\epsilon . The gradient for a single sample is g ^ = − x ⁡ ( x ⊤ ​ Δ ​ θ + ϵ ) \hat{g}=-x(x^{\top}\Delta\theta+\epsilon) , where Δ ​ θ = θ ⋆ − θ \Delta\theta=\theta_{\star}-\theta . The batch gradient is the average g ¯ n = 1 B ​ ∑ b = 1 B g ^ n , b \bar{g}_{n}=\frac{1}{B}\sum_{b=1}^{B}\hat{g}_{n,b} . Since samples within a batch are i.i.d., the covariance of the batch gradient scales as: Cov ⁡ ( g ¯ n ) = 1 B ​ Cov ​ ( g ^ n ) = 1 B ​ ( 𝔼 ⁡ [ g ^ n ​ g ^ n ⊤ ∣ θ n ] − 𝔼 ⁡ [ g ^ n ] ​ 𝔼 ​ [ g ^ n ] ⊤ ) . \mathrm{Cov}(\bar{g}_{n})=\frac{1}{B}\mathrm{Cov}(\hat{g}_{n})=\frac{1}{B}\left(\mathbb{E}[\hat{g}_{n}\hat{g}_{n}^{\top}\mid\theta_{n}]-\mathbb{E}[\hat{g}_{n}]\mathbb{E}[\hat{g}_{n}]^{\top}\right). Expanding the single-sample second moment as an outer product: 𝔼 ⁡ [ g ^ n ​ g ^ n ⊤ ∣ θ n ] \displaystyle\mathbb{E}[\hat{g}_{n}\hat{g}_{n}^{\top}\mid\theta_{n}] = 𝔼 ⁡ [ ( x ​ x ⊤ ​ Δ ​ θ + x ​ ϵ ) ​ ( x ​ x ⊤ ​ Δ ​ θ + x ​ ϵ ) ⊤ ] \displaystyle=\mathbb{E}\left[(xx^{\top}\Delta\theta+x\epsilon)(xx^{\top}\Delta\theta+x\epsilon)^{\top}\right] = 𝔼 ⁡ [ x ​ x ⊤ ​ ( Δ ​ θ ​ Δ ​ θ ⊤ ) ​ x ​ x ⊤ ] + 𝔼 ⁡ [ x ​ ϵ 2 ​ x ⊤ ] . \displaystyle=\mathbb{E}[xx^{\top}(\Delta\theta\Delta\theta^{\top})xx^{\top}]+\mathbb{E}[x\epsilon^{2}x^{\top}]. Using 𝔼 ⁡ [ ϵ ] = 0 \mathbb{E}[\epsilon]=0 and 𝔼 ⁡ [ ϵ 2 ] = σ 2 \mathbb{E}[\epsilon^{2}]=\sigma^{2} , the covariance becomes: Cov ⁡ ( g ^ n ) = σ 2 ​ H + 𝔼 ⁡ [ x ​ x ⊤ ​ ( Δ ​ θ ​ Δ ​ θ ⊤ ) ​ x ​ x ⊤ ] − H ⁡ ( Δ ​ θ ​ Δ ​ θ ⊤ ) ​ H ⏟ State-dependent noise . \mathrm{Cov}(\hat{g}_{n})=\sigma^{2}H+\underbrace{\mathbb{E}[xx^{\top}(\Delta\theta\Delta\theta^{\top})xx^{\top}]-H(\Delta\theta\Delta\theta^{\top})H}_{\text{State-dependent noise}}. The state-dependent noise decays toward zero as the agent converges ( Δ ​ θ → 0 \Delta\theta\to 0 ). Assuming this state-dependent noise is negligible compared to the irreducible noise floor at convergence, Cov ⁡ ( g ^ n ) ≈ σ 2 ​ H \mathrm{Cov}(\hat{g}_{n})\approx\sigma^{2}H , the batch covariance is σ 2 B ​ H \frac{\sigma^{2}}{B}H .

Plugging this into the Lyapunov equation from Lemma 19 : H ​ Σ ∞ + Σ ∞ ​ H = η ​ σ 2 B ​ H . H\Sigma_{\infty}+\Sigma_{\infty}H=\frac{\eta\sigma^{2}}{B}H. Since H H and Σ ∞ \Sigma_{\infty} must commute, we obtain Σ ∞ = η ​ σ 2 2 ​ B ​ I \Sigma_{\infty}=\frac{\eta\sigma^{2}}{2B}I . ∎

###### Lemma 21 (JTL Gradient Noise Covariance) .

Consider a Joint-Task Learning (JTL) agent optimizing a mixture of tasks with weights { w j k } j = 1 k \{w_{j}^{k}\}_{j=1}^{k} . Let H j H_{j} be the Hessian of task j j and θ ⋆ j \theta_{\star}^{j} its optimum. The stationary gradient noise covariance is: C ∞ JTL = 1 B ( 𝔼 j [ σ j 2 H j ] + 𝔼 j [ ∇ ℒ j ( θ ⋆ JTL , k ) ∇ ℒ j ( θ ⋆ JTL , k ) ⊤ ] ) . C^{\text{JTL}}_{\infty}=\frac{1}{B}\left(\mathbb{E}_{j}[\sigma_{j}^{2}H_{j}]+\mathbb{E}_{j}\left[\nabla\mathcal{L}^{j}(\theta_{\star}^{\text{JTL},k})\nabla\mathcal{L}^{j}(\theta_{\star}^{\text{JTL},k})^{\top}\right]\right).

###### Proof.

Recall that the mixture optimum is defined as: θ ⋆ JTL , k = ( ∑ j = 1 k w j k ​ H j ) − 1 ​ ( ∑ j = 1 k w j k ​ H j ​ θ ⋆ j ) . \theta_{\star}^{\text{JTL},k}=\left(\sum_{j=1}^{k}w_{j}^{k}H_{j}\right)^{-1}\left(\sum_{j=1}^{k}w_{j}^{k}H_{j}\theta_{\star}^{j}\right). (25) At the mixture optimum θ ⋆ JTL , k \theta_{\star}^{\text{JTL},k} , the expected gradient of a sampled task j j is g j = H j ​ ( θ ⋆ JTL , k − θ ⋆ j ) g_{j}=H_{j}(\theta_{\star}^{\text{JTL},k}-\theta_{\star}^{j}) . By definition of the optimum, the weighted sum of these gradients is zero: 𝔼 j ​ [ g j ] = ∑ w j k ​ g j = 0 \mathbb{E}_{j}[g_{j}]=\sum w_{j}^{k}g_{j}=0 . Using the Law of Total Variance for the batch gradient noise C n JTL C^{\text{JTL}}_{n} : C n JTL = 1 B ​ ( 𝔼 j ​ [ Cov ξ j ​ ( g ^ n j ) ] + Cov j ​ ( 𝔼 ⁡ [ g ^ n j ] ) ) . C^{\text{JTL}}_{n}=\frac{1}{B}\left(\mathbb{E}_{j}[\mathrm{Cov}_{\xi^{j}}(\hat{g}^{j}_{n})]+\mathrm{Cov}_{j}(\mathbb{E}[\hat{g}^{j}_{n}])\right). (26) By Lemma 20 , for each task, Cov ⁡ ( g ^ j ∣ j ) = σ j 2 ​ H j \mathrm{Cov}(\hat{g}_{j}\mid j)=\sigma_{j}^{2}H_{j} . Because the mean expected gradient is zero, the variance of the conditional expectations is simply the second moment: Cov j ​ ( g j ) = 𝔼 j ​ [ g j ​ g j ⊤ ] = ∑ j = 1 k w j k ​ [ H j ​ ( θ ⋆ JTL , k − θ ⋆ j ) ​ ( θ ⋆ JTL , k − θ ⋆ j ) ⊤ ​ H j ] . \mathrm{Cov}_{j}(g_{j})=\mathbb{E}_{j}[g_{j}g_{j}^{\top}]=\sum_{j=1}^{k}w_{j}^{k}\left[H_{j}(\theta_{\star}^{\text{JTL},k}-\theta_{\star}^{j})(\theta_{\star}^{\text{JTL},k}-\theta_{\star}^{j})^{\top}H_{j}\right]. (27) Thus, noting that g j = ∇ ℒ j ​ ( θ ⋆ JTL , k ) g_{j}=\nabla\mathcal{L}^{j}(\theta_{\star}^{\text{JTL},k}) , the total noise covariance at convergence is: C ∞ JTL = 1 B ( 𝔼 j [ σ j 2 H j ] + 𝔼 j [ ∇ ℒ j ( θ ⋆ JTL , k ) ∇ ℒ j ( θ ⋆ JTL , k ) ⊤ ] ) . C^{\text{JTL}}_{\infty}=\frac{1}{B}\left(\mathbb{E}_{j}[\sigma_{j}^{2}H_{j}]+\mathbb{E}_{j}\left[\nabla\mathcal{L}^{j}(\theta_{\star}^{\text{JTL},k})\nabla\mathcal{L}^{j}(\theta_{\star}^{\text{JTL},k})^{\top}\right]\right). ∎

##### A.4.2 JTL Geometry

For JTL, deriving the instability and transient cost requires careful handling of two distinct geometries: the curvature driving the optimization dynamics (the mixture Hessian, H dyn H_{\text{dyn}} ) and the curvature determining the evaluation cost (the target task Hessian, H eval H_{\text{eval}} ).

To obtain a tractable closed-form solution that preserves geometric intuition, we rely on a task alignment assumption.

###### Assumption 22 (Task Alignment) .

We assume the Hessians of all tasks are simultaneously diagonalizable. This implies that while tasks may vary in difficulty (eigenvalues), their principal axes of curvature (eigenvectors) are aligned. Consequently, the driving and evaluation Hessians commute: H dyn ​ H eval = H eval ​ H dyn . H_{\text{dyn}}H_{\text{eval}}=H_{\text{eval}}H_{\text{dyn}}. (28)

To interpret the cost geometrically, we introduce the Relative Curvature Matrix , denoted by ℛ \mathcal{R} . This matrix measures how steep the evaluation landscape is relative to the driving landscape: ℛ := H dyn − 1 ​ H eval . \mathcal{R}:=H_{\text{dyn}}^{-1}H_{\text{eval}}. (29) The eigenvalues of ℛ \mathcal{R} provide a scaling factor for the cost. If ℛ ≺ I \mathcal{R}\prec I , the agent benefits from a pre-conditioning effect, effectively training on a steeper surrogate than the test task.

We also require the following assumption, standard in the analysis of constant step-size SGD ( Mandt et al., 2017 ; Bach and Moulines, 2013 ) .

###### Assumption 23 (Bounded Curvature) .

We assume the Hessian matrices for all tasks have eigenvalues bounded strictly away from zero and infinity. There exist constants 0 < λ min ≤ λ max 0<\lambda_{\min}\leq\lambda_{\max} such that for any task k k : λ min ​ I ⪯ H k ⪯ λ max ​ I . \lambda_{\min}I\preceq H^{k}\preceq\lambda_{\max}I. Consequently, the mixture Hessian H dyn = ∑ j = 1 k w j k ​ H j H_{\text{dyn}}=\sum_{j=1}^{k}w_{j}^{k}H^{j} satisfies the same spectral bounds.

##### A.4.3 Exact Instability

We now quantify the Instability ℐ k \mathcal{I}^{k} , defined as the difference in asymptotic performance between the Joint-Task Learning (JTL) and Independent-Task Learning (ITL) agents on task k k .

###### Theorem 24 (Exact Instability Decomposition) .

Let the JTL agent optimize a mixture of tasks with weights { w j k } j ≤ k \{w_{j}^{k}\}_{j\leq k} , converging in mean to the mixture optimum θ ⋆ JTL , k \theta_{\star}^{\text{JTL},k} . Assuming the observation noise is uniform across tasks ( σ j 2 ≈ σ 2 \sigma_{j}^{2}\approx\sigma^{2} ) and tasks are aligned ( Assumption 22 ), the Instability gap on task k k decomposes exactly into: ℐ k ≈ ℒ k ​ ( θ ⋆ JTL , k ) − ℒ k ​ ( θ ⋆ k ) ⏟ Approximation Bias + η 4 ​ B ​ ∑ j = 1 k w j k ​ ‖ ∇ ℒ j ​ ( θ ⋆ JTL , k ) ‖ ℛ k 2 ⏟ Task Diversity . \mathcal{I}^{k}\approx\underbrace{\mathcal{L}^{k}(\theta_{\star}^{\text{JTL},k})-\mathcal{L}^{k}(\theta_{\star}^{k})}_{\text{Approximation Bias}}+\frac{\eta}{4B}\underbrace{\sum_{j=1}^{k}w_{j}^{k}\,\|\nabla\mathcal{L}^{j}(\theta_{\star}^{\text{JTL},k})\|^{2}_{\mathcal{R}^{k}}}_{\text{Task Diversity}}. (30)

###### Proof.

The instability ℐ k \mathcal{I}^{k} is the difference between the asymptotic risks of the JTL and ITL agents. Using the decomposition in ( 23 ), we analyze the bias and variance components separately.

First, consider the bias . The ITL agent is unbiased for task k k , as its mean converges exactly to θ ⋆ k \theta_{\star}^{k} . In contrast, the JTL agent converges in mean to the stationary point of the mixture loss: θ ⋆ JTL , k = ( ∑ j = 1 k w j k ​ H j ) − 1 ​ ( ∑ j = 1 k w j k ​ H j ​ θ ⋆ j ) . \theta_{\star}^{\text{JTL},k}=\left(\sum_{j=1}^{k}w_{j}^{k}H_{j}\right)^{-1}\left(\sum_{j=1}^{k}w_{j}^{k}H_{j}\theta_{\star}^{j}\right). This results in a permanent bias penalty of 1 2 ​ ‖ θ ⋆ JTL , k − θ ⋆ k ‖ H k 2 = ℒ k ​ ( θ ⋆ JTL , k ) − ℒ k ​ ( θ ⋆ k ) \frac{1}{2}\|\theta_{\star}^{\text{JTL},k}-\theta_{\star}^{k}\|_{H_{k}}^{2}=\mathcal{L}^{k}(\theta_{\star}^{\text{JTL},k})-\mathcal{L}^{k}(\theta_{\star}^{k}) .

Second, consider the variance . The variance contribution to the expected loss of task k k is proportional to the trace of the Hessian and the stationary parameter covariance Σ ∞ \Sigma_{\infty} , i.e., 1 2 ​ Tr ​ ( H k ​ Σ ∞ ) \frac{1}{2}\text{Tr}(H_{k}\Sigma_{\infty}) . By Lemma 20 , for the ITL agent this is simply η ​ σ 2 4 ​ B ​ Tr ​ ( H k ) \frac{\eta\sigma^{2}}{4B}\text{Tr}(H_{k}) .

For the JTL agent, the stationary covariance Σ ∞ JTL , k \Sigma^{\text{JTL},k}_{\infty} satisfies the Lyapunov equation: H dyn ​ Σ ∞ JTL , k + Σ ∞ JTL , k ​ H dyn = η ​ C ∞ JTL , k . H_{\text{dyn}}\Sigma^{\text{JTL},k}_{\infty}+\Sigma^{\text{JTL},k}_{\infty}H_{\text{dyn}}=\eta C^{\text{JTL},k}_{\infty}. (31) Although the gradient covariance term C ∞ JTL , k C^{\text{JTL},k}_{\infty} does not necessarily share an eigenbasis with H dyn H_{\text{dyn}} , H k H_{k} and H dyn H_{\text{dyn}} commute under Assumption 22 . This allows us to apply the trace identity for Lyapunov equations ( Tr ​ ( M ​ Σ ) = 1 2 ​ Tr ​ ( M ​ H − 1 ​ η ​ C ) \text{Tr}(M\Sigma)=\frac{1}{2}\text{Tr}(MH^{-1}\eta C) ) to calculate the loss contribution directly: 1 2 ​ Tr ​ ( H k ​ Σ ∞ JTL , k ) = η 4 ​ Tr ​ ( H k ​ H dyn − 1 ​ C ∞ JTL , k ) = η 4 ​ Tr ​ ( ℛ k ​ C ∞ JTL , k ) . \frac{1}{2}\text{Tr}(H_{k}\Sigma^{\text{JTL},k}_{\infty})=\frac{\eta}{4}\text{Tr}\left(H_{k}H_{\text{dyn}}^{-1}C^{\text{JTL},k}_{\infty}\right)=\frac{\eta}{4}\text{Tr}\left(\mathcal{R}^{k}C^{\text{JTL},k}_{\infty}\right). (32) Notice that by definition ℛ k = H dyn − 1 ​ H k \mathcal{R}^{k}=H_{\text{dyn}}^{-1}H_{k} . Because the task Hessians commute, ℛ k \mathcal{R}^{k} is symmetric and H dyn − 1 ​ H k = H k ​ H dyn − 1 = ℛ k H_{\text{dyn}}^{-1}H_{k}=H_{k}H_{\text{dyn}}^{-1}=\mathcal{R}^{k} . Substituting the JTL gradient noise covariance from Lemma 21 : η 4 ​ B ​ Tr ​ ( ℛ k ​ 𝔼 j ​ [ σ j 2 ​ H j ] ) + η 4 ​ B ​ Tr ​ ( ℛ k ​ 𝔼 j ​ [ g j ​ g j ⊤ ] ) , \frac{\eta}{4B}\text{Tr}\left(\mathcal{R}^{k}\mathbb{E}_{j}[\sigma_{j}^{2}H_{j}]\right)+\frac{\eta}{4B}\text{Tr}\left(\mathcal{R}^{k}\mathbb{E}_{j}[g_{j}g_{j}^{\top}]\right), (33) where g j = ∇ ℒ j ​ ( θ ⋆ JTL , k ) g_{j}=\nabla\mathcal{L}^{j}(\theta_{\star}^{\text{JTL},k}) . Under the assumption of uniform observation noise ( σ j 2 ≈ σ 2 \sigma_{j}^{2}\approx\sigma^{2} ), the first term yields η ​ σ 2 4 ​ B ​ Tr ​ ( ℛ k ​ H dyn ) = η ​ σ 2 4 ​ B ​ Tr ​ ( H k ) \frac{\eta\sigma^{2}}{4B}\text{Tr}(\mathcal{R}^{k}H_{\text{dyn}})=\frac{\eta\sigma^{2}}{4B}\text{Tr}(H_{k}) , which exactly cancels the ITL variance floor in the instability gap.

The remaining term is the Task Diversity penalty. Using the linearity of expectation and the trace property Tr ​ ( A ​ u ​ u ⊤ ) = u ⊤ ​ A ​ u \text{Tr}(Auu^{\top})=u^{\top}Au : η 4 ​ B ​ 𝔼 j ​ [ Tr ​ ( ℛ k ​ g j ​ g j ⊤ ) ] = η 4 ​ B ​ ∑ j = 1 k w j k ​ g j ⊤ ​ ℛ k ​ g j = η 4 ​ B ​ ∑ j = 1 k w j k ​ ‖ ∇ ℒ j ​ ( θ ⋆ JTL , k ) ‖ ℛ k 2 , \frac{\eta}{4B}\mathbb{E}_{j}\left[\text{Tr}(\mathcal{R}^{k}g_{j}g_{j}^{\top})\right]=\frac{\eta}{4B}\sum_{j=1}^{k}w_{j}^{k}g_{j}^{\top}\mathcal{R}^{k}g_{j}=\frac{\eta}{4B}\sum_{j=1}^{k}w_{j}^{k}\|\nabla\mathcal{L}^{j}(\theta_{\star}^{\text{JTL},k})\|^{2}_{\mathcal{R}^{k}}, (34) completing the decomposition. ∎

##### A.4.4 Exact Transient Error

We now derive the explicit form of the Transient Error δ N \delta_{N} . This metric captures the cumulative cost of learning as the agent travels from its initialization to its state at step N N .

###### Theorem 25 (The Transient Potential Law) .