To Retain or to Adapt? Generalizing Continual Learning

arxiv:2607.05609

Original source · Paper JSON

Post-training · 2026-07-06

Cites (3) · Cited by (0)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4
5. Part 5
6. Part 6
7. Part 7
8. Part 8
9. Part 9
10. Part 10
11. Part 11
12. Part 12
13. Part 13
14. Part 14
15. Part 15

Contents

• To Retain or to Adapt? Generalizing Continual Learning
• Abstract
• 1 Introduction
• Contributions.
• 2 Background & Setup
• 2.1 A General Objective for Continual Learning
• General setting.
• Continual learning objective.
• 2.2 Analysis Setup and Formalism
• Definition 1 (Task-based data generating process.) .
• 2.3 The algorithm as a Markov Process
• 2.4 JTL and ITL processes
• The Joint-Task Learning algorithm ( 𝒜 JTL \mathcal{A}_{\text{JTL}} ).
• The Independent-Task Learning algorithm ( 𝒜 ITL \mathcal{A}_{\text{ITL}} ).
• Equivalent computational costs.
• 3 Transfer Efficiency
• Definition 2 (Transfer Efficiency) .
• Definition 3 (Instability) .
• Definition 4 (Transient Error) .
• Lemma 5 (TE Decomposition) .
• 3.1 The Critical Task Duration
• Lemma 6 (The Critical Task Duration) .
• 3.2 Empirical Validation of the Framework
• (P1) Controlled Instability: Permuted- and Shuffled-CIFAR.
• (P2) Crossing the Critical Task Duration in CLEAR.
• 4 Quadratic-Loss Analysis
• 4.1 Setup
• 4.2 Closed-form Instability and Transient Error
• 4.3 Asymptotic Conditions for Positive Transfer
• Definition 7 (Growth Rate) .
• Theorem 8 (Asymptotic behaviour of Transfer for Quadratic Objectives — Informal) .
• 4.4 The Continuous Drift Model
• 4.5 The Optimization Bias Matters
• 5 Generalizing CL beyond the JTL/ITL dichotomy
• 5.1 From implicit to explicit future models
• Definition 9 (Predictive CL Algorithm) .
• 5.2 Proof of Concept: the Window Predictive CL Algorithm
• Definition 10 (Mean Squared Displacement (MSD)) .
• Lemma 11 (Asymptotic Optimality of Window Size) .
• Beyond the Window: richer future models.
• 6 Interdisciplinary Literature Review
• Online Learning and Stochastic Optimization.
• Multi-Task Learning (MTL).
• Learning To Learn and Online Meta-Learning.
• Continual Learning.
• 7 Discussion and Conclusion
• Limitations.
• Acknowledgements
• Appendix
• Appendix A Theoretical proofs
• A.1 Introduction to the Framework of Kumar et al. (2025)
• The Environment.
• Agent and Learning Algorithm.
• Continual Learning Objective.
• Supervised Specialization.
• A.2 Analysis Setup and Formalism
• Task-Based Formulation.
• Definition (Task-Based Data Generating Process) .
• Batched Observations.
• The Data-Generating Process.
• Measures of Error and Performance.
• A.2.1 The Algorithm as a Markov Process
• Convergence.
• Stochastic Gradient Descent Instantiations.
• ITL SGD ( 𝒜 ITL \mathcal{A}_{\text{ITL}} ).
• JTL SGD ( 𝒜 JTL \mathcal{A}_{\text{JTL}} ).
• A.2.2 Why use reset ITL as the reference?
• A.3 Decomposition: Instability and Transient Error
• Definition 12 (Task-wise Convergence) .
• Definition 13 (Instability ℐ k \mathcal{I}^{k} ) .
• Definition 14 (Transient Error δ k \delta^{k} ) .
• Lemma 15 (Exact TE Decomposition) .
• Proof.
• Lemma 16 (The Critical Task Duration) .
• Proof.
• On the Sub-Linear Transient Error Assumption.
• A.4 Exact Analysis in the Quadratic Regime
• Assumption 17 (Quadratic Tasks) .
• A.4.1 SGD Dynamics on Quadratic Landscapes
• Lemma 18 (Convergence of the Mean) .
• Proof.
• Lemma 19 (Parameter Covariance) .
• Proof.
• Lemma 20 (ITL Gradient Noise and Parameter Covariance) .
• Proof.
• Lemma 21 (JTL Gradient Noise Covariance) .
• Proof.
• A.4.2 JTL Geometry
• Assumption 22 (Task Alignment) .
• Assumption 23 (Bounded Curvature) .
• A.4.3 Exact Instability
• Theorem 24 (Exact Instability Decomposition) .
• Proof.
• A.4.4 Exact Transient Error
• Theorem 25 (The Transient Potential Law) .
• Geometric Interpretation.
• Proof.
• A.5 Asymptotic Dynamics of the First and Second Order Moments
• Definition 26 (Growth Rate) .
• A.5.1 Geometric Drift
• Lemma 27 (Geometric Mean Drift) .
• Proof.
• Lemma 28 (Geometric Covariance Drift) .
• Proof.
• Lemma 29 (Expected Asymptotic Instability) .
• Proof.
• A.5.2 Residual Errors
• Lemma 30 (Contraction of Residuals) .
• Proof.
• Lemma 31 (Asymptotic Dominance of Stable Recursions) .
• Proof.
• A.5.3 Asymptotic Transfer
• Theorem 32 (Asymptotic Condition for Positive Transfer Efficiency) .
• Proof.
• A.6 (Non-asymptotic) Continuous Drift Analysis in the Quadratic Model
• A.6.1 Setup and Definitions
• A.6.2 Derivation of Expected Instability
• A.6.3 Derivation of Expected Transient Error
• ITL Agent.
• JTL Agent.
• A.6.4 Non-asymptotic Condition for Positive Transfer
• A.7 Window Algorithm as an Instance of Predictive CL
• A.7.1 Asymptotic Analysis in the Quadratic Loss Setting
• Definition 33 (Mean Squared Displacement) .
• Remark (Relationship between MSD and Growth Rate) .
• A.7.2 Window Geometric Drift
• Lemma 34 (Window Geometric Drift) .
• Proof.
• Lemma 35 (Window Covariance Drift) .
• Proof.
• Lemma 36 (Asymptotic Instability of the Window Agent) .
• Proof.
• A.7.3 Asymptotically Optimal Window
• Lemma 37 (Asymptotic Optimality of Window Size) .
• Proof.
• Case 1: Bounded Regime.
• Case 2: Linear Drift Regime.
• Case 3: Diffusive Regime.
• A.7.4 Window Agent for Continuously Drifting Tasks
• A.7.5 Instability of the Window Agent
• Centroid Lag.
• Diversity Noise.
• A.7.6 Transient Error of the Window Agent
• A.7.7 The Optimal Window W ⋆ W^{\star}
• Appendix B Empirical Setup
• B.1 Simulations
• B.1.1 Shallow Linear Experiments
• B.1.2 Deep Linear Experiments
• B.1.3 Window Predictive CL Algorithm Experiments
• Real-Time Trajectory Evaluation with W ⋆ W^{\star} :
• Transfer Efficiency Landscapes:
• B.2 Real-World Benchmarks
• B.2.1 CLEAR
• B.2.2 MULTIDATASET (MD5)
• B.2.3 Meta-World
• B.3 Synthetic Benchmarks
• B.3.1 Permuted CIFAR-10
• B.3.2 Shuffled CIFAR-10
• B.4 Training Procedures & Networks
• Supervised Learning
• Reinforcement Learning
• B.4.1 Hyperparameters
• Supervised Learning
• Reinforcement Learning
• B.5 Additional Figures and Empirical Substantiation
• B.5.1 Pairwise Transfer is Asymmetric, and Why Ordering Matters
• B.5.2 Proxy measures of instability
• B.5.3 Scaling Deep Linear Networks Experiments
• B.5.4 Real-time training trajectories
• Instructions for reporting errors