To Retain or to Adapt? Generalizing Continual Learning — part 11/15

Cleaned-text code points [154774, 170703). Bibliography omitted here; source.md includes it.

Simultaneously, the JTL agent’s covariance tracks the equilibrium covariance of the centroid. Because the gradient noise increases as tasks spread out, this target Σ ∞ JTL , k \Sigma_{\infty}^{\text{JTL},k} grows over time. The noise covariance C JTL , k C^{\text{JTL},k} at the centroid is dominated by the task scatter ( Lemma 21 ): C JTL , k = 1 B ​ σ ξ 2 ​ H + 1 k ​ B ​ ∑ j = 1 k H ​ 𝔼 ​ [ ( θ ⋆ JTL , k − θ ⋆ j ) ​ ( θ ⋆ JTL , k − θ ⋆ j ) ⊤ ] ​ H . C^{\text{JTL},k}=\frac{1}{B}\sigma_{\xi}^{2}H+\frac{1}{kB}\sum_{j=1}^{k}H\mathbb{E}[(\theta_{\star}^{\text{JTL},k}-\theta_{\star}^{j})(\theta_{\star}^{\text{JTL},k}-\theta_{\star}^{j})^{\top}]H. Using the scatter result from the instability derivation, the summation approximates to k 2 6 ​ Σ Δ \frac{k^{2}}{6}\Sigma_{\Delta} . Thus: C JTL , k ≈ 1 B ​ σ ξ 2 ​ H + k 6 ​ B ​ H ​ Σ Δ ​ H . C^{\text{JTL},k}\approx\frac{1}{B}\sigma_{\xi}^{2}H+\frac{k}{6B}H\Sigma_{\Delta}H. The equilibrium covariance satisfies the Lyapunov equation H ​ Σ ∞ JTL , k + Σ ∞ JTL , k ​ H = η ​ C JTL , k H\Sigma_{\infty}^{\text{JTL},k}+\Sigma_{\infty}^{\text{JTL},k}H=\eta C^{\text{JTL},k} . Taking the trace (and exploiting the cyclic property Tr ​ ( H ​ Σ Δ ​ H ) = Tr ​ ( H 2 ​ Σ Δ ) \text{Tr}(H\Sigma_{\Delta}H)=\text{Tr}(H^{2}\Sigma_{\Delta}) ): 2 ​ Tr ​ ( H ​ Σ ∞ JTL , k ) = η B ​ σ ξ 2 ​ Tr ​ ( H ) + η ​ k 6 ​ B ​ Tr ​ ( H 2 ​ Σ Δ ) . 2\text{Tr}(H\Sigma_{\infty}^{\text{JTL},k})=\frac{\eta}{B}\sigma_{\xi}^{2}\text{Tr}(H)+\frac{\eta k}{6B}\text{Tr}(H^{2}\Sigma_{\Delta}). The only quantity dependent on k k is the second term. The covariance drift is the difference between consecutive targets: Tr ​ ( H ⁡ ( Σ ∞ JTL , k − 1 − Σ ∞ JTL , k ) ) = − η 12 ​ B ​ Tr ​ ( H 2 ​ Σ Δ ) . \text{Tr}\left(H(\Sigma_{\infty}^{\text{JTL},k-1}-\Sigma_{\infty}^{\text{JTL},k})\right)=-\frac{\eta}{12B}\text{Tr}(H^{2}\Sigma_{\Delta}). This drift is constant with respect to k k . The initialization error Σ ~ 0 JTL , k \tilde{\Sigma}_{0}^{\text{JTL},k} tracks this drift. By the contraction property ( Lemma 30 ) with rate ρ Σ < 1 \rho_{\Sigma}<1 : Tr ​ ( H ​ Σ ~ 0 JTL , k ) ≤ − Tr ​ ( H ⁡ ( Σ ∞ JTL , k − 1 − Σ ∞ JTL , k ) ) + ρ Σ ​ Tr ​ ( H ​ Σ ~ 0 JTL , k − 1 ) . \text{Tr}(H\tilde{\Sigma}_{0}^{\text{JTL},k})\leq-\text{Tr}(H(\Sigma_{\infty}^{\text{JTL},k-1}-\Sigma_{\infty}^{\text{JTL},k}))+\rho_{\Sigma}\text{Tr}(H\tilde{\Sigma}_{0}^{\text{JTL},{k-1}}). Because the target variance is strictly increasing, the agent is constantly "catching up," meaning Tr ​ ( H ​ Σ ~ 0 JTL , k − 1 ) \text{Tr}(H\tilde{\Sigma}_{0}^{\text{JTL},{k-1}}) is consistently negative and acts to further pull the trace down. Thus, we can safely bound: Tr ​ ( H ​ Σ ~ 0 JTL , k ) ≤ − η 12 ​ B ​ Tr ​ ( H 2 ​ Σ Δ ) . \text{Tr}(H\tilde{\Sigma}_{0}^{\text{JTL},k})\leq-\frac{\eta}{12B}\text{Tr}(H^{2}\Sigma_{\Delta}). Given the boundedness of H H ( Assumption 23 ), we approximate Tr ​ ( Σ ~ 0 JTL , k ) ≤ − η 12 ​ B ​ Tr ​ ( H ​ Σ Δ ) \text{Tr}(\tilde{\Sigma}_{0}^{\text{JTL},k})\leq-\frac{\eta}{12B}\text{Tr}(H\Sigma_{\Delta}) .

##### A.6.4 Non-asymptotic Condition for Positive Transfer

We investigate the conditions under which JTL outperforms ITL. We define the Positive Transfer Regime as the set of environment parameters for which the total expected error of the JTL agent is strictly lower than that of the ITL agent: 𝔼 ⁡ [ ℐ JTL k ] + 𝔼 ⁡ [ δ JTL k ] < 𝔼 ⁡ [ δ ITL k ] . \mathbb{E}[\mathcal{I}^{k}_{\text{JTL}}]+\mathbb{E}[\delta^{k}_{\text{JTL}}]<\mathbb{E}[\delta^{k}_{\text{ITL}}]. (93) Denote by σ Δ 2 = Tr ​ ( Σ Δ ) \sigma^{2}_{\Delta}=\text{Tr}(\Sigma_{\Delta}) the drift magnitude parameter. Let c μ = 1 1 − ρ μ c_{\mu}=\frac{1}{1-\rho_{\mu}} represent the effective time constant. Substituting the derived expected bounds, we obtain the master inequality for positive transfer in continuous drift settings: γ min ​ ( k ​ σ Δ 2 + 2 ​ σ 0 2 − η ​ σ ξ 2 / 2 ​ B ) τ ​ N ⏟ ITL Transient Cost > k 6 ​ Tr ​ ( Σ x ​ Σ Δ ) + η ​ k 24 ​ B ​ Tr ​ ( Σ x 2 ​ Σ Δ ) ⏟ JTL Instability Cost + γ max τ ​ N ​ ( c μ 2 ​ σ Δ 2 3 ​ k − η 12 ​ B ​ Tr ​ ( Σ x ​ Σ Δ ) ) ⏟ JTL Transient Cost . \underbrace{\frac{\gamma_{\min}\,(k\sigma^{2}_{\Delta}+2\sigma_{0}^{2}-\eta\sigma_{\xi}^{2}/2B)}{\tau N}}_{\text{ITL Transient Cost}}>\underbrace{\frac{k}{6}\text{Tr}(\Sigma_{x}\Sigma_{\Delta})+\frac{\eta k}{24B}\text{Tr}(\Sigma_{x}^{2}\Sigma_{\Delta})}_{\text{JTL Instability Cost}}+\underbrace{\frac{\gamma_{\max}}{\tau N}\left(c_{\mu}^{2}\frac{\sigma^{2}_{\Delta}}{3k}-\frac{\eta}{12B}\text{Tr}(\Sigma_{x}\Sigma_{\Delta})\right)}_{\text{JTL Transient Cost}}. (94)

Notice that this inequality precisely recovers the more general asymptotic result from Theorem 32 : as k → ∞ k\to\infty , both the ITL Transient cost and the JTL Instability cost grow linearly in k k , while the JTL Transient cost is lower order in k k . Therefore, these two linear terms eventually dominate the balance between positive and negative transfer in the limit of infinite tasks.

#### A.7 Window Algorithm as an Instance of Predictive CL

We introduce the Window algorithm as a concrete instance of our generalized CL framework: a Predictive CL algorithm that bridges both the ITL and JTL approaches through the definition of a window of tasks . At any task k k , the Window agent predicts the relevant future distribution with the recent W W tasks and optimizes the average loss over that window, where we call W W the window size . Specifically, the sampling distribution 𝒟 W k \mathcal{D}^{k}_{W} is: 𝒟 W k = ∑ j = k − W + 1 k ρ j k ​ 𝒫 j ρ j k = | N j | ∑ i ∈ [ k − W , k ] N i . \mathcal{D}^{k}_{W}=\sum_{j=k-W+1}^{k}\rho_{j}^{k}\,\mathcal{P}^{j}\qquad\rho_{j}^{k}=\frac{|N_{j}|}{\sum_{i\in[k-W,k]}N_{i}}. Like the JTL agent, the Window agent’s parameters are not reset at the beginning of a new task. Clearly, as W W grows, the Window agent converges to the JTL agent—which can be viewed as an infinite window agent—whereas setting W = 1 W=1 recovers a variant of the ITL agent with warm starts .

##### A.7.1 Asymptotic Analysis in the Quadratic Loss Setting

In contrast to the JTL agent, whose stability is determined by the global growth of the task solutions relative to the origin, the Window agent’s stability depends entirely on the local variations of the environment over the horizon W W . To characterize this, we utilize the Mean Squared Displacement (MSD) measure.

###### Definition 33 (Mean Squared Displacement) .

Let m : ℕ × ℕ → ℝ + m:\mathbb{N}\times\mathbb{N}\to\mathbb{R}^{+} be a function characterizing the expected squared Euclidean distance between two task solutions separated by a lag W W at time k k : 𝔼 ⁡ [ ‖ θ ⋆ k − θ ⋆ k − W ‖ 2 ] ∈ Θ ⁡ ( m ⁡ ( k , W ) ) . \mathbb{E}[\|\theta_{\star}^{k}-\theta_{\star}^{k-W}\|^{2}]\in\Theta(m(k,W)). (95)

We analyze the three canonical regimes under the assumption of stationary increments. The Mean Squared Displacement scales as m ⁡ ( W ) ∈ Θ ⁡ ( 1 ) m(W)\in\Theta(1) for Bounded, Θ ⁡ ( W ) \Theta(W) for Diffusive, and Θ ⁡ ( W 2 ) \Theta(W^{2}) for Linear environments.

###### Remark (Relationship between MSD and Growth Rate) .

The Mean Squared Displacement is universally upper-bounded by the global growth rate: m ⁡ ( k , W ) ∈ 𝒪 ⁡ ( ψ ⁡ ( k ) ) m(k,W)\in\mathcal{O}(\psi(k)) . This follows from the triangle inequality relative to the origin. However, for the standard regimes of interest (Diffusive and Linear), the process exhibits stationary increments . In these cases, m ⁡ ( k , W ) m(k,W) depends only on the lag W W and is independent of the absolute time k k .

##### A.7.2 Window Geometric Drift

The geometric drift for the Window agent is defined by the update to the sliding centroid and the shift in the stationary covariance matrix: Δ ​ μ W , k \displaystyle\Delta\mu^{W,k} = ( θ ⋆ W , k − 1 − θ ⋆ W , k ) , \displaystyle=(\theta_{\star}^{W,k-1}-\theta_{\star}^{W,k}), (96) Δ ​ Σ W , k \displaystyle\Delta\Sigma^{W,k} = ( Σ ∞ W , k − 1 − Σ ∞ W , k ) . \displaystyle=(\Sigma_{\infty}^{W,k-1}-\Sigma_{\infty}^{W,k}). (97) Table 5 summarizes the asymptotic scaling of these drift terms. Unlike the JTL agent, where the drift is suppressed by the infinitely growing history k k , the Window agent’s drift is governed by the ratio of the Mean Squared Displacement to the constant window inertia W W .

We now present a quadratic loss analysis of the Transfer Efficiency of the Window agent compared to the ITL agent. The analysis follows the same steps as the JTL derivation, re-using several previous results.

As shown previously, the transient cost is dominated by the geometric drift of the agent’s optimum: Δ ​ μ W , k = θ ⋆ W , k − θ ⋆ W , k − 1 \Delta\mu^{W,k}=\theta_{\star}^{W,k}-\theta_{\star}^{W,k-1} . At step k k , the minimum of the Window agent objective is: θ ⋆ W , k = ( ∑ j = k − W k ρ j k ​ H j ) − 1 ​ ( ∑ j = k − W k ρ j k ​ H j ​ θ ⋆ j ) . \theta_{\star}^{W,k}=\left(\sum_{j=k-W}^{k}\rho_{j}^{k}H_{j}\right)^{-1}\left(\sum_{j=k-W}^{k}\rho_{j}^{k}H_{j}\theta_{\star}^{j}\right).

###### Lemma 34 (Window Geometric Drift) .

For a Window agent of size W W , the squared magnitude of the geometric mean shift scales with the growth rate of the window interval: 𝔼 ⁡ [ ‖ Δ ​ μ W , k ‖ 2 ] ∈ Θ ⁡ ( m ⁡ ( k , W ) W 2 ) . \mathbb{E}[\|\Delta\mu^{W,k}\|^{2}]\in\Theta\left(\frac{m(k,W)}{W^{2}}\right). (98)

###### Proof.

Let H ¯ k = 1 W ​ ∑ j ∈ 𝒲 k H j \bar{H}_{k}=\frac{1}{W}\sum_{j\in\mathcal{W}_{k}}H_{j} be the average Hessian. The optimum satisfies H ¯ k ​ θ ⋆ W , k = 1 W ​ ∑ j ∈ 𝒲 k H j ​ θ ⋆ j \bar{H}_{k}\theta_{\star}^{W,k}=\frac{1}{W}\sum_{j\in\mathcal{W}_{k}}H_{j}\theta_{\star}^{j} .

Comparing steps k k and k − 1 k-1 , the window 𝒲 k \mathcal{W}_{k} is obtained from 𝒲 k − 1 \mathcal{W}_{k-1} by removing task k − W k-W and adding task k k . The average Hessian updates as: H ¯ k = H ¯ k − 1 + 1 W ​ ( H k − H k − W ) . \bar{H}_{k}=\bar{H}_{k-1}+\frac{1}{W}(H_{k}-H_{k-W}). The weighted target vector v k = H ¯ k ​ θ ⋆ W , k v_{k}=\bar{H}_{k}\theta_{\star}^{W,k} updates as: v k = v k − 1 + 1 W ​ ( H k ​ θ ⋆ k − H k − W ​ θ ⋆ k − W ) . v_{k}=v_{k-1}+\frac{1}{W}(H_{k}\theta_{\star}^{k}-H_{k-W}\theta_{\star}^{k-W}). Substituting v k − 1 = H ¯ k − 1 ​ θ ⋆ W , k − 1 v_{k-1}=\bar{H}_{k-1}\theta_{\star}^{W,k-1} and solving for the difference: H ¯ k ​ θ ⋆ W , k = ( H ¯ k − 1 W ​ ( H k − H k − W ) ) ​ θ ⋆ W , k − 1 + 1 W ​ ( H k ​ θ ⋆ k − H k − W ​ θ ⋆ k − W ) . \bar{H}_{k}\theta_{\star}^{W,k}=\left(\bar{H}_{k}-\frac{1}{W}(H_{k}-H_{k-W})\right)\theta_{\star}^{W,k-1}+\frac{1}{W}(H_{k}\theta_{\star}^{k}-H_{k-W}\theta_{\star}^{k-W}). Rearranging terms to isolate the drift Δ ​ μ W , k = θ ⋆ W , k − θ ⋆ W , k − 1 \Delta\mu^{W,k}=\theta_{\star}^{W,k}-\theta_{\star}^{W,k-1} : H ¯ k ​ ( θ ⋆ W , k − θ ⋆ W , k − 1 ) = 1 W ​ [ H k ​ ( θ ⋆ k − θ ⋆ W , k − 1 ) − H k − W ​ ( θ ⋆ k − W − θ ⋆ W , k − 1 ) ] . \bar{H}_{k}(\theta_{\star}^{W,k}-\theta_{\star}^{W,k-1})=\frac{1}{W}\left[H_{k}(\theta_{\star}^{k}-\theta_{\star}^{W,k-1})-H_{k-W}(\theta_{\star}^{k-W}-\theta_{\star}^{W,k-1})\right]. Multiplying by H ¯ k − 1 \bar{H}_{k}^{-1} : Δ ​ μ W , k = 1 W ​ H ¯ k − 1 ​ [ H k ​ ( θ ⋆ k − θ ⋆ W , k − 1 ) + H k − W ​ ( θ ⋆ W , k − 1 − θ ⋆ k − W ) ] . \Delta\mu^{W,k}=\frac{1}{W}\bar{H}_{k}^{-1}\left[H_{k}(\theta_{\star}^{k}-\theta_{\star}^{W,k-1})+H_{k-W}(\theta_{\star}^{W,k-1}-\theta_{\star}^{k-W})\right]. This expression represents the pull of the new task k k and the "release" of the old task k − W k-W on the centroid. Taking the squared norm expectation and applying the bound on the condition number κ \kappa ( Assumption 23 ): 𝔼 ⁡ [ ‖ Δ ​ μ W , k ‖ 2 ] ≤ κ 2 W 2 ​ 𝔼 ​ [ ‖ ( θ ⋆ k − θ ⋆ W , k − 1 ) + ( θ ⋆ W , k − 1 − θ ⋆ k − W ) ‖ 2 ] . \mathbb{E}[\|\Delta\mu^{W,k}\|^{2}]\leq\frac{\kappa^{2}}{W^{2}}\mathbb{E}\left[\left\|(\theta_{\star}^{k}-\theta_{\star}^{W,k-1})+(\theta_{\star}^{W,k-1}-\theta_{\star}^{k-W})\right\|^{2}\right]. The term inside the norm is dominated by the displacement between the incoming and outgoing tasks, ‖ θ ⋆ k − θ ⋆ k − W ‖ \|\theta_{\star}^{k}-\theta_{\star}^{k-W}\| . By the definition of the Mean Squared Displacement ( Definition 33 ), the expected squared distance between two tasks separated by W W steps scales as Θ ⁡ ( m ⁡ ( k , W ) ) \Theta(m(k,W)) . Thus: 𝔼 ⁡ [ ‖ Δ ​ μ W , k ‖ 2 ] ∈ Θ ⁡ ( m ⁡ ( k , W ) W 2 ) . \mathbb{E}[\|\Delta\mu^{W,k}\|^{2}]\in\Theta\left(\frac{m(k,W)}{W^{2}}\right). ∎

###### Lemma 35 (Window Covariance Drift) .

The geometric covariance shift for the Window agent scales asymptotically with the finite difference of the Mean Squared Displacement: | 𝔼 ⁡ [ Tr ⁡ ( Δ ​ Σ W , k ) ] | ∈ Θ ⁡ ( Δ ​ m ​ ( k , W ) ) . |\mathbb{E}[\mathrm{Tr}(\Delta\Sigma^{W,k})]|\in\Theta(\Delta m(k,W)). (99)

Implication: Unlike the JTL agent, the Window agent achieves a stable second moment (constant uncertainty size) because the task diversity within the active window does not grow infinitely with time k k .

###### Proof.

Following the logic of Lemma 28 , the stationary covariance Σ ∞ \Sigma_{\infty} scales with the task diversity 𝒱 div \mathcal{V}_{\text{div}} . For the Window agent, this diversity is measured over the set 𝒲 k \mathcal{W}_{k} : 𝒱 div W , k = 1 W ​ ∑ j ∈ 𝒲 k ‖ ∇ ℒ j ​ ( θ ⋆ W , k ) ‖ 2 = 1 W ​ ∑ j ∈ 𝒲 k ‖ θ ⋆ j − θ ⋆ W , k ‖ H j 2 2 . \mathcal{V}_{\text{div}}^{W,k}=\frac{1}{W}\sum_{j\in\mathcal{W}_{k}}\|\nabla\mathcal{L}^{j}(\theta_{\star}^{W,k})\|^{2}=\frac{1}{W}\sum_{j\in\mathcal{W}_{k}}\|\theta_{\star}^{j}-\theta_{\star}^{W,k}\|_{H_{j}^{2}}^{2}. This term represents the spatial variance of W W consecutive tasks. The covariance shift is: 𝔼 ⁡ [ Tr ⁡ ( Δ ​ Σ W , k ) ] \displaystyle\mathbb{E}[\mathrm{Tr}(\Delta\Sigma^{W,k})] = 𝔼 ⁡ [ Tr ⁡ ( C ∞ W , k − 1 − C ∞ W , k ) ] \displaystyle=\mathbb{E}[\mathrm{Tr}(C^{W,k-1}_{\infty}-C^{W,k}_{\infty})] (100) = 1 W ​ ( ∑ j ∈ 𝒲 k − 1 𝔼 ⁡ [ ‖ ∇ ℒ j ​ ( θ ⋆ W , k − 1 ) ‖ 2 ] − ∑ j ∈ 𝒲 k 𝔼 ⁡ [ ‖ ∇ ℒ j ​ ( θ ⋆ W , k ) ‖ 2 ] ) . \displaystyle=\frac{1}{W}\left(\sum_{j\in\mathcal{W}_{k-1}}\mathbb{E}[\|\nabla\mathcal{L}^{j}(\theta_{\star}^{W,k-1})\|^{2}]-\sum_{j\in\mathcal{W}_{k}}\mathbb{E}[\|\nabla\mathcal{L}^{j}(\theta_{\star}^{W,k})\|^{2}]\right). (101)

Recognizing that 𝔼 ⁡ [ ‖ ∇ ℒ j ​ ( θ ⋆ W , k − 1 ) ‖ 2 ] = 𝔼 ⁡ [ ‖ θ ⋆ j − θ ⋆ W , k − 1 ‖ H j 2 2 ] \mathbb{E}[\|\nabla\mathcal{L}^{j}(\theta_{\star}^{W,k-1})\|^{2}]=\mathbb{E}[\|\theta_{\star}^{j}-\theta_{\star}^{W,k-1}\|_{H_{j}^{2}}^{2}] , which is the expected norm of the average difference among the task optima in 𝒲 k − 1 \mathcal{W}_{k-1} , we know by Definition 33 that this scales as Θ ⁡ ( m ⁡ ( k − 1 , W ) ) \Theta(m(k-1,W)) . Similarly, 𝔼 ⁡ [ ‖ ∇ ℒ j ​ ( θ ⋆ W , k ) ‖ 2 ] ∈ Θ ⁡ ( m ⁡ ( k , W ) ) \mathbb{E}[\|\nabla\mathcal{L}^{j}(\theta_{\star}^{W,k})\|^{2}]\in\Theta(m(k,W)) . Thus, asymptotically, the difference between these two scales as Θ ⁡ ( m ⁡ ( k − 1 , W ) ) − Θ ⁡ ( m ⁡ ( k , W ) ) = : Δ ​ m ​ ( k , W ) \Theta(m(k-1,W))-\Theta(m(k,W))=:\Delta m(k,W) .

Overall, taking the absolute value of the trace, we have: | 𝔼 ⁡ [ Tr ⁡ ( Δ ​ Σ W , k ) ] | ∈ Θ ⁡ ( Δ ​ m ​ ( k , W ) ) . |\mathbb{E}[\mathrm{Tr}(\Delta\Sigma^{W,k})]|\in\Theta(\Delta m(k,W)). ∎

###### Lemma 36 (Asymptotic Instability of the Window Agent) .

The expected Instability cost ℐ W \mathcal{I}^{W} for the Window agent, which comprises the approximation bias and the stationary gradient noise, scales linearly with the Mean Squared Displacement: 𝔼 ⁡ [ ℐ W ] ∈ Θ ⁡ ( m ⁡ ( k , W ) ) . \mathbb{E}[\mathcal{I}^{W}]\in\Theta(m(k,W)). (102)

###### Proof.

The proof follows the same structure as Lemma 29 .

The instability for the Window agent is: ℐ W ≈ ( ℒ k ​ ( θ ⋆ W , k ) − ℒ k ​ ( θ ⋆ k ) ) ⏟ Bias + η 4 ​ B ​ ∑ j = k − W k ρ j k ​ ‖ ∇ ℒ j ​ ( θ ⋆ W , k ) ‖ ℛ k 2 ⏟ Variance . \mathcal{I}^{W}\approx\underbrace{\left(\mathcal{L}^{k}(\theta_{\star}^{W,k})-\mathcal{L}^{k}(\theta_{\star}^{k})\right)}_{\text{Bias}}+\underbrace{\frac{\eta}{4B}\sum_{j=k-W}^{k}\rho_{j}^{k}\,\|\nabla\mathcal{L}^{j}(\theta_{\star}^{W,k})\|_{\mathcal{R}^{k}}^{2}}_{\text{Variance}}.