Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy β part 4/9
Cleaned-text code points [46529, 62020). Bibliography omitted here; source.md includes it.
Within the Distance-based category, ? (?) propose two primary approaches: i) fixed data representation; and ii) learning representation. In the former, a fixed embedding generator is used in every Incremental Learning step and the major challenge is to ensure that the generated embeddings are representative of all upcoming classes in a way that keeps embeddings from samples belonging to the same class grouped together but apart from embeddings from samples of other classes. In the latter, the embedding generator also suffers adjustment in its parameters in each training step. The main challenge is the tracking of the embedding region of the classes when the embedding generator updates and the model does not have access to samples from those classes.
The use of prototype classes is a common technique employed in many methods within the Distance-based category. A prototype class is an embedding that represents the central tendency of other samples from the same class. Figure 6(a) provides an example where the embeddings are colored light yellow and light red, representing cats and dogs, respectively. The dark yellow point ( p Β― c subscript Β― π π \overline{p}_{c} overΒ― start_ARG italic_p end_ARG start_POSTSUBSCRIPT italic_c end_POSTSUBSCRIPT ) is the prototype class of cats and is created by averaging the features of the embedding of the cats. Similarly, the dark red point ( p Β― d subscript Β― π π \overline{p}_{d} overΒ― start_ARG italic_p end_ARG start_POSTSUBSCRIPT italic_d end_POSTSUBSCRIPT ) is the prototype class of dogs and is also created by averaging the features of its class.
The use of a prototype class, instead of all samples, helps Incremental Learning to reduce memory requirements and avoid training with classes with unbalanced sample amounts. The first problem is handled because the model has to store only one data point. The second problem was noticed by ? (?). They showed that the models suffer from a bias in favor of new classes. This bias is caused by the number of samples of new classes in relation to the old ones.
The choice of embedding representation is crucial in determining how well a classifier can avoid forgetting. When the representations of samples from different tasks are aligned in embedding space, learning one task facilitates the learning of the other. Conversely, when they are orthogonal, learning one task does not affect the learning of the other. In a recent study, ? (?) used a meta-learning algorithm to train a feature extractor that searches for representations that maximize parallel or orthogonal embeddings, thus enabling the classifier to determine a region for each taskβs classes.
In a few-shot learning setting, aligning visual embedding and word embedding is another technique for choosing the space for each class in the embedding space. ? (?) proposed a method where the model takes an image as input and produces an embedding that should be close to the word embedding that represents the class, instead of generating a label as output. They tested this method with Word2Vec embedding (?) and GloVe embedding (?). By doing so, a model that can handle natural language classification problems can be used without experiencing CF.
In situations where the embedding generator updates its weights during training, the embedding space of a class can change. To address this issue, Semantic Drift Compensation (SDC) (?) updates all prototypes at the end of each training session to discover the new region in the embedding space where a class will be allocated. To calculate this change, the model determines the position of the incoming data in the embedding space before training and again at the end of the session. The difference between these positions is then used to update the prototypes. However, itβs important to note that SDC assumes that all class embeddings continue to be clustered with their respective prototypes, which is not guaranteed. Therefore, while this technique can be effective in certain scenarios, it may not always provide accurate updates to the prototypes.
Incremental Unsupervised Learning of representations using the MixUp technique (?) has yielded promising results that surpass those obtained with Supervised Incremental Learning. One possible explanation for this phenomenon is that unsupervised learning generates a smooth loss landscape (?). This approach has been tested with Barlow Twins (?) and SimSiam (?). The main classifier used in this category is based on cosine similarity. Two improvements have been proposed for this classifier: (i) generating multiple perspectives of each sample, and (ii) using a Contrastive loss. In the first approach, each sample is used to generate multiple views of itself by applying rotations. In the second approach, the generated samples are used as anchors in a Contrastive loss (?).
When a model is trained using the Joint Training approach, where all classes are trained together, it leads to a better decision boundary within the embedding space compared to training in an incremental fashion. In general, this approach tends to produce a more robust class separation, effectively enhancing the modelβs ability to distinguish between different classes and improving overall generalization performance. Therefore, ? (?) propose an objective function that aims to enforce the decision boundary generated through incremental training to mimic that of Joint Training.
Figure 8 provides an overview of the Distance-based category. One of its advantages over other categories is that it can expand its known classes without changing its structure, focusing only on the embedding space. However, it remains unclear how many classes an embedding space can effectively support without causing confusion. In the next section, we will introduce a category that attempts to address this issue by identifying sub-modules within the model that can mitigate CF.
### 3.3 Sub-networks
When learning a new task, previously learned patterns are replaced by the current training patterns, which can lead to forgetting. For example, a model may use a neuron connection as a positive stimulus to improve its performance when learning its initial task, and later, when learning a second task, that same neuron may need to be used as a negative stimulus, which can decrease the modelβs performance in the previous task. To address this issue, some approaches try to encode the knowledge needed for each task in separate sections of the model with minimal overlap, as shown in Figure 6(b) . By considering a neural network as a high-capacity model with redundant connections, it becomes feasible to selectively update specific subsets of neurons during the training of each new task. These subsets, known as Sub-networks, consist of distinct sets of neurons within the main model, and the aim is to minimize their overlap.
? (?) led a pioneering study on the subject of CF. They conducted a series of experiments with various activation functions and regularization techniques to determine which combinations were less prone to CF. They found that the combination of the ReLU (?) activation function and Dropout (?) regularization was the most effective. Dropout works by randomly disabling a subset of connections during each training iteration, which forces the model to learn using the remaining active connections. This encourages patterns to be encoded redundancy in different subsets of connections, meaning that even if some connections change in future tasks, there may still be alternative paths using different subsets of connections that can solve the initial task. This suggests that a model can be composed of multiple sub-models. Therefore, choosing a good training regimen can reduce CF in a wide range of models (?), such as the best combination of dropout, weight decay, learning rate, batch size, and optimizer. Itβs important to note that these training regimens can also be used in conjunction with other techniques discussed in this survey.
The Sub-network category can be classified into two types: Soft Sub-networks and Hard Sub-networks. Hard Sub-network-based approaches aim to identify a subset of parameters that represent the learned knowledge of a specific task and then freeze them. Consequently, when the model has to learn a new task, it adjusts a different subset of neurons to form small networks within the model. This produces a submodel for each learned task, each with varying performance. However, because a new subset of parameters must be frozen for each trained task, the number of tasks the model can learn is limited, as the overall network size remains constant.
Like Hard Sub-networks, Soft Sub-networks also select a set of important parameters for task t i subscript π‘ π t_{i} italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT . However, instead of keeping these parameters frozen by default, a term is added to the loss function that penalizes changes to the parameters of task t i subscript π‘ π t_{i} italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT . This ensures that the network will do its best to avoid changing the important parameters of task t i subscript π‘ π t_{i} italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT while generating values for the parameters of task t i + 1 subscript π‘ π 1 t_{i+1} italic_t start_POSTSUBSCRIPT italic_i + 1 end_POSTSUBSCRIPT . In this way, Soft Sub-networks try to overcome the capacity limitations observed in Hard Sub-networks approaches by allowing neurons to suffer little changes during training new tasks.
Adding a penalization term to the loss function changes the loss landscape. It has been established that in the loss landscape, multiple local minima can yield satisfactory levels of accuracy for a specific task, as illustrated in Figure 9 (?). By adding a penalty term, each submodel can be forced to reach a local minimum that provides optimal performance for each task simultaneously. With Soft Sub-networks, different local minima can correspond to different solvable tasks for each submodel. Therefore, the goal of successful Soft Sub-network training is to bring the local minima of each submodel closer together, allowing the parameters of one submodel to be reused by another submodel. For example, after learning task t i subscript π‘ π t_{i} italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT and starting to learn task t i + 1 subscript π‘ π 1 t_{i+1} italic_t start_POSTSUBSCRIPT italic_i + 1 end_POSTSUBSCRIPT , the network is forced to find an acceptable solution for task t i + 1 subscript π‘ π 1 t_{i+1} italic_t start_POSTSUBSCRIPT italic_i + 1 end_POSTSUBSCRIPT that is as close as possible to the solution found for task t i subscript π‘ π t_{i} italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT . The works that most contributed to these two approaches will be discussed in the following subsections.
#### 3.3.1 Soft Sub-networks
In the Soft Sub-networks, each submodel specializes in solving a task that shares parameters (weights) with the other submodels, which helps to avoid the capacity limitations of Hard Sub-networks. However, the subset of important parameters for task t i subscript π‘ π t_{i} italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , which will be shared by the submodels of subsequent tasks t i + 1 , t i + 2 , β¦ subscript π‘ π 1 subscript π‘ π 2 β¦ {t_{i+1},t_{i+2},\dots} italic_t start_POSTSUBSCRIPT italic_i + 1 end_POSTSUBSCRIPT , italic_t start_POSTSUBSCRIPT italic_i + 2 end_POSTSUBSCRIPT , β¦ , cannot go through a lot of changes when those tasks are being trained. To keep these parameters useful for the original task, they must be identified prior to beginning the training of task t i + j subscript π‘ π π t_{i+j} italic_t start_POSTSUBSCRIPT italic_i + italic_j end_POSTSUBSCRIPT , where j π j italic_j represents a positive integer value. The Fisher Information Matrix (FIM) was used by ? (?) to determine which parameters are most important for solving task t i subscript π‘ π t_{i} italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT . This information is then incorporated into the loss function as a regularization term to prevent changes to these parameters during the training of task t i + 1 subscript π‘ π 1 t_{i+1} italic_t start_POSTSUBSCRIPT italic_i + 1 end_POSTSUBSCRIPT . This method is called Elastic Weight Consolidation (EWC) and is one of the most widely cited approaches in the field of Soft Sub-networks.
? (?) introduced Predictive EWC as an enhancement to the existing EWC algorithm. In Predictive EWC, before the model starts learning a new task t i + 1 subscript π‘ π 1 t_{i+1} italic_t start_POSTSUBSCRIPT italic_i + 1 end_POSTSUBSCRIPT , it initially processes the new training data using the current model. This pre-processing step enables the model to identify and discard samples that it can handle accurately, retaining only those with lower accuracy. Consequently, a refined data set is formed, based on the assumption that certain samples are no longer necessary for the model since it has already learned how to handle them effectively. This refinement of the data set leads to a reduction in training time.
In a similar vein, ? (?) suggests that when data from different tasks exhibit substantial distribution differences, a more aggressive regularization approach should be applied to important parameters. To address this concern, the author proposes a conditional loss called Check Regularization, which facilitates parameter grouping based on each task, while still allowing parameter sharing between similar tasks. By incorporating Check Regularization, the model can adapt its regularization strategy to account for variations in task distributions, ultimately improving performance across diverse tasks.
The EWC algorithm assumes that the FIM is a diagonal matrix. However, in practice, this assumption does not hold true. To address this limitation, a technique proposed by ? (?) can be employed. The idea is to rotate the modelβs parameters in such a way that: i) the modelβs output remains unchanged; and, ii) the FIM calculated from gradients becomes approximately diagonal. This rotation-based approach has demonstrated superior accuracy compared to the original EWC method. However, it comes at the cost of increased processing time.
While these approaches have shown improvements, the use of strong regularization techniques can potentially hinder the modelβs ability to acquire new knowledge. Additionally, there are two notable drawbacks associated with these methods. Firstly, they do not consider the presence of batch normalization layers, which are commonly used in DNN. Secondly, they involve a high computational cost during post-processing when calculating the FIM. These drawbacks should be taken into consideration when applying these techniques in practical scenarios.
In CF research, the influence of batch normalization layers is often overlooked due to their potential to degrade model accuracy when the data distribution changes over time, rather than improving it. However, to tackle this particular drawback, a promising solution called the Continual Normalization (CN) layer has been proposed by ? (?). The CN layer performs spatial normalization on the feature map using group normalization. By incorporating CN layers into CF models, the adverse effects of batch normalization on changing data distributions can be mitigated, thereby preserving model accuracy over time.