Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy — part 1/9

Cleaned-text code points [0, 15291). Bibliography omitted here; source.md includes it.

HTML conversions sometimes display errors due to content that did not convert correctly from the source. This paper uses the following packages that are not yet supported by the HTML conversion tool. Feedback on these issues are not necessary; they are known and are being worked on.

failed: rawfonts

Authors: achieve the best HTML results from your LaTeX submissions by selecting from this list of supported packages .

# Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy

###### Abstract

Deep Learning models have achieved remarkable performance in tasks such as image classification or generation, often surpassing human accuracy. However, they can struggle to learn new tasks and update their knowledge without access to previous data, leading to a significant loss of accuracy known as Catastrophic Forgetting (CF). This phenomenon was first observed by McCloskey and Cohen in 1989 and remains an active research topic. Incremental learning without forgetting is widely recognized as a crucial aspect in building better AI systems, as it allows models to adapt to new tasks without losing the ability to perform previously learned ones. This article surveys recent studies that tackle CF in modern Deep Learning models that use gradient descent as their learning algorithm. Although several solutions have been proposed, a definitive solution or consensus on assessing CF is yet to be established. The article provides a comprehensive review of recent solutions, proposes a taxonomy to organize them, and identifies research gaps in this area.

## 1 Introduction

Incremental Learning, also known as Continual Learning, is the ability to progressively learn new tasks, one at a time, without forgetting the previously learned tasks, and also use the accumulated knowledge to facilitate learning future tasks. For humans, the process of learning a new task can be simpler when it is related to an already-known task. For instance, learning to ride a motorcycle should be simpler if you already know how to ride a bicycle. This property is called Forward Knowledge Transfer (?), and this is a desirable property for any AI system. Forward Knowledge is widely known as transfer learning with fine-tuning in Neural Networks (?, ?). However, the problem with transfer learning approaches is that prior knowledge is forgotten when learning a new task, causing a drastic reduction in performance on previous tasks. Backward knowledge propagation is another important human characteristic that is desirable in an AI incremental learning system (?). This property expects that after learning a task related to a previous one, the performance of the previous one will also increase or at least be maintained. When these properties fail and there is a high decrease in performance on the previous tasks in favor of the new one, the model is said to suffer from Catastrophic Forgetting (CF).

? (?) showed that the biggest challenge for AI systems to support Incremental Learning is the CF phenomenon. It was studied in many types of models, for instance, Support Vector Machine (SVM) (?), however, this issue is more notable in connectionist models (?), and this is mainly caused by the way the parameters of these models are adjusted. When a new pattern is presented to the model, these parameters — also known as weights — change in value, losing the model’s capability to detect previous patterns. There is still no agreement on how to handle this issue in AI today, making it an open problem.

Since the mid-2000s, connectionist models known as Deep Learning (DL) (?), have become dominant in Machine Learning. The most common DL models are those constructed from multilayer Artificial Neural Networks (ANNs), Convolution Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Transformers, which we also refer to as Deep Neural Networks (DNN). Although several DNN models have been proposed, including models that can outperform humans in numerous tasks, such models continue to suffer from the effects of CF on Incremental Learning. The Incremental Learning problem is formulated in several ways, some accepting more relaxed constraints and others trying to better approximate real-world constraints (?, ?).

In a recent study by ? (?), it was demonstrated that the problem of determining the optimal parameters to avoid the CF in a fixed model can be reduced to the well-known Satisfiability (SAT) problem (?). Consequently, this proof categorizes the CF problem within the class of NP-HARD problems. Essentially, each potential arrangement of weights, denoted as θ t subscript 𝜃 𝑡 \theta_{t} italic_θ start_POSTSUBSCRIPT italic_t end_POSTSUBSCRIPT , within a model can be seen as an element in a set of solutions that satisfy a given task t 𝑡 t italic_t . Thus, the key to solving the CF problem lies in identifying a specific combination of θ 𝜃 \theta italic_θ values that exist at the intersection of the solution sets for all tasks. By reducing CF to an NP-HARD problem, it becomes evident that effectively addressing this challenge is crucial for advancements in artificial intelligence in the years to come. In Figure 1 , we can visualize the set of all possible parameter combinations, denoted as Θ Θ \Theta roman_Θ , for a given model. The subsets θ 1 subscript 𝜃 1 \theta_{1} italic_θ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , θ 2 subscript 𝜃 2 \theta_{2} italic_θ start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT , and θ 3 subscript 𝜃 3 \theta_{3} italic_θ start_POSTSUBSCRIPT 3 end_POSTSUBSCRIPT represent the parameter combinations that individually solve tasks 1, 2, and 3, respectively. Furthermore, there are smaller subsets, such as θ 12 subscript 𝜃 12 \theta_{12} italic_θ start_POSTSUBSCRIPT 12 end_POSTSUBSCRIPT , θ 23 subscript 𝜃 23 \theta_{23} italic_θ start_POSTSUBSCRIPT 23 end_POSTSUBSCRIPT , and θ 13 subscript 𝜃 13 \theta_{13} italic_θ start_POSTSUBSCRIPT 13 end_POSTSUBSCRIPT , that solve combinations of two tasks simultaneously. Finally, the set θ 123 subscript 𝜃 123 \theta_{123} italic_θ start_POSTSUBSCRIPT 123 end_POSTSUBSCRIPT represents the parameter combination that simultaneously solves all three tasks. It is worth noting that finding this specific combination becomes increasingly challenging as it involves satisfying the requirements of multiple tasks simultaneously, leading to a combinatorial problem.

Despite the NP-HARD nature of CF, significant progress has been made in mitigating CF within Deep Neural Network (DNN) models through various techniques and heuristics. Several studies have proposed effective solutions, including ? (?, ?, ?, ?, ?, ?, ?, ?, ?, ?). In this paper, we present a comprehensive examination of these techniques, shedding light on the current state of research in this domain.

Due to the absence of a clear consensus on how to evaluate these models numerically, we refrain from direct numerical comparisons. Instead, our focus lies in discussing the most suitable application scenarios for each technique and comprehensively assessing their respective benefits and drawbacks. This approach allows for a more nuanced understanding of the strengths and limitations of each method, aiding researchers and practitioners in making informed decisions regarding their implementation in real-world settings. The primary contributions of this research endeavor can be concisely encapsulated in the following threefold:

• proposes a taxonomy to cluster techniques to avoid CF in DNN models based on the strategy;

• explores the progression of techniques within each category, starting from 2012 with the rise in popularity of DNNs;

• and, discuss the strengths and weaknesses inherent in each category, fostering a comprehensive understanding of their respective merits and limitations.

The rest of this work is organized as follows: in Section 2 of this article, we present related articles that empirically contrast various strategies for preventing catastrophic forgetting (?, ?, ?, ?); in Section 3 we propose a taxonomy for DNN models categorized into four main groups: Rehearsal, Distance-Based, Dynamic Networks, and Sub-Networks, according to their learning strategies. These models can be used in many contexts, such as computer vision, natural language processing, and others; and, finally, Section 4 brings final considerations.

## 2 Related work

? (?) carry out a review on aspects of Incremental Learning with Neural Networks, which also details techniques related to Incremental Learning. Although this work was published recently, several new techniques, insights, and solutions have emerged since then. Therefore, we use a systematic review approach to fully map the latest emerging techniques. We follow the guidelines given by ? (?) to retrieve and filter works considered state-of-the-art in this area. We intend to present and explain the methods published since 2012, as it is the period of greatest attention for DNN models.

The current state of research on CF is limited in that many surveys only compare empirical findings or are confined to a single experimental setup. For example, recent surveys by ? (?) and ? (?) have only focused on specific aspects of continual learning. However, ? (?) conducted an extensive study on task incremental learning setups for image classification, where the model is aware of the task it is evaluating. They proposed a taxonomy that categorizes solutions into three categories: Replay, Regularization-based, and Parameter isolation methods, each with two or three sub-categories. Notably, this taxonomy does not include hybrid or distance-based methods, which could be an interesting avenue for future research.

The work by ? (?) provides a comprehensive analysis of fixed-size models in the context of image classification. Their study focuses specifically on class incremental learning, a setup in which the model has no knowledge of which task an input belongs to when evaluating, but each task introduces new classes. To handle this, models typically create new output neurons for each new class. The authors demonstrate that a straightforward fine-tuning approach using a memory of past tasks, also known as a Replay Buffer, can achieve competitive accuracy compared to more complex methods.

In their study, ? (?) introduce a novel setup called domain-incremental learning, which they investigate in the context of computer vision. In this setup, each new task involves inputs from new classes related to previous ones. For example, the first task may involve detecting cats in images of cats in a garden in the morning, while the second task involves detecting cats in images of cats indoors. The problem remains the same, but the domain changes.

We not only cover computer vision but also a diverse range of topics, such as Natural Language Processing (NLP) and Reinforcement Learning (RL), employing a variety of models including fully connected networks, Generative Neural Networks (GAN), and others. Our survey also presents three setups that have gained attention in online and few-shot learning: (i) Online Incremental Learning, (ii) Unbounded Task Incremental Learning, and (iii) Data-free Incremental Learning, which have stricter constraints and require methods that are specific to these setups. Previous surveys have not covered these setups, and as demonstrated by ? (?), methods not designed for online learning are inadequate for these setups.

The survey of ? (?) shows the four main reasons why CF occurs: i) weight drift, which occurs when the model adjusts its parameters to new tasks; ii) activation drift, similar to the former, but just in the classification layer; iii) inter-task confusion, which occurs when samples from different tasks have similar features; and finally, iv) task-recency bias, which occurs when the model is biased toward the last task trained on. It is important to emphasize that these reasons are valid for the setup of class incremental learning that allow the storage of a part of the data set from previous tasks.

In the context of Natural Language Processing (NLP), there are also efforts to train models incrementally. ? (?) examined and compared methods used in various NLP applications. In turn, ? (?) compares the resilience of pretrained language models to avoid CF in a sequence of tasks using several techniques. However, unlike us, they do not fit current methods into a taxonomy. Instead, they present an evaluation of methods in continuous learning scenarios.

As ? (?) demonstrated, comparing results between models can be influenced by various variables such as task order or chosen setup. Therefore, our goal is not to provide another empirical survey but rather to offer a comprehensive overview of the field from multiple perspectives. To achieve this, we will introduce a taxonomy that categorizes existing methods, allowing for easier comparison between them. In the following section, we will present the most relevant works and organize them according to this taxonomy. Our aim is to provide a broad understanding of the area and understand the appropriate usage of them.

## 3 Taxonomic Organization

To provide a clear framework for organizing methods that prevent CF, we suggest categorizing them into four main groups: rehearsal, distance-based, sub-networks, and dynamic networks. Figure 2 visually represents this taxonomy.

A method falls under the Rehearsal category if it utilizes data from previously learned tasks, such as a replay buffer in RL. This can include artificial data or representations of learned knowledge, like embeddings from prior tasks. Models that employ similarity estimates based on the distance between samples to determine task or class membership are classified as Distance-based . For instance, samples projected on a hyperplane can be considered part of the same class. The Sub-networks group comprises models that aim to prevent knowledge or parameter overlap between tasks. One approach is to divide the model into smaller sub-models, each with its own set of weights, which are used to learn one task. Two sub-models can also share some components. Dynamic Networks , on the other hand, refer to models whose structure, such as the number of nodes or layers, expands to accommodate additional tasks. As more tasks are learned, the model’s capacity increases, allowing it to recognize more patterns. In the following sections, we will explore each of these categories in greater detail.

### 3.1 Rehearsal

Rehearsal is the process of repeating a previous task while learning a new one to prevent forgetting. This technique is commonly used in human psychology and was introduced in the study of neural networks by Robins ? (?, ?). However, storing all of the data that a model is presented with throughout its lifespan and using it to retrain the model can be impractical, as it requires a large amount of storage space and can significantly increase the amount of time required for training. To address these issues, some researchers have focused on reducing the amount of data that needs to be stored, while others have focused on reducing the training time (?, ?, ?, ?). These approaches are known as mini-rehearsal and pseudo-rehearsal and will be discussed in the following subsections.

#### 3.1.1 Pseudo-rehearsal