Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy — part 6/9

Cleaned-text code points [77087, 93041). Bibliography omitted here; source.md includes it.

Some tasks can share a mask, or at least a part of it, without causing the CF problem on past tasks while allowing the new task to be learned. It works on tasks that are similar. ? (?) define that tasks t i subscript 𝑡 𝑖 t_{i} italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT and t i + j subscript 𝑡 𝑖 𝑗 t_{i+j} italic_t start_POSTSUBSCRIPT italic_i + italic_j end_POSTSUBSCRIPT , where j 𝑗 j italic_j represents a positive integer value, are considered similar if a positive knowledge transfer from t i subscript 𝑡 𝑖 t_{i} italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT to t i + j subscript 𝑡 𝑖 𝑗 t_{i+j} italic_t start_POSTSUBSCRIPT italic_i + italic_j end_POSTSUBSCRIPT is observed. For this, a model trained from scratch in task t i + j subscript 𝑡 𝑖 𝑗 t_{i+j} italic_t start_POSTSUBSCRIPT italic_i + italic_j end_POSTSUBSCRIPT is compared with a pretrained model in task t i subscript 𝑡 𝑖 t_{i} italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT fine-tuned to task t i + j subscript 𝑡 𝑖 𝑗 t_{i+j} italic_t start_POSTSUBSCRIPT italic_i + italic_j end_POSTSUBSCRIPT . If the fine-tuned model has a better performance they consider that a positive knowledge transfer has been observed. They also used HAT to find the masks.

Piggyback (?) represents a departure from the above approach of selecting a subset of parameters for training specific to task t i 𝑖 {}_{i} start_FLOATSUBSCRIPT italic_i end_FLOATSUBSCRIPT . The Piggyback method involves finding a mask that, when applied to a frozen pretrained model, such as a ResNet (?) that has been trained on the ImageNet (?) data set, results in a solution for task t i 𝑖 {}_{i} start_FLOATSUBSCRIPT italic_i end_FLOATSUBSCRIPT . The mask is used to decide which parameters of the model should be used for each task; however, the model weights are never changed. When the backbone is trained on a task in a different context than the tasks being evaluated, the results are not satisfactory as the new patterns cannot be learned. ? (?) extend this concept to GANs, creating a bank of filters that the model can use, instead of creating, in future tasks. Recently, ? (?) show that these masks can also be used in models from Vision in Transformers (ViT) to avoid the CF.

Modular networks consist of multiple clusters, or modules, comprising densely interconnected neurons that have sparse connections with neurons in other clusters (?, ?, ?). In the context of RL, modular networks have been employed to address CF (?). However, there is a lack of comparative results against established benchmarks, as the authors proposed a novel RL problem. They designed a distinct set of input neurons for each task, as depicted in Figure 11 . The red neurons receive samples exclusively from one task, while the blue neurons receive samples exclusively from the other task. This arrangement enables the internal structure of the model to selectively activate different subsets of neurons for each task.

A variation of modular networks, known as Diffusion-based Neuro-modulation, was introduced by ? (?) to identify task-specific modules within the model. The authors established connection points within the model to regulate modularity. For their evaluation, two additional input points were introduced, one for each task. The first point received a positive stimulus, while the stimulus at the other point was suppressed when processing samples from the first task. Conversely, the stimulus was inverted when processing samples from the second task. The authors solely tested their method on the specific problem they proposed, and its extension to other tasks is challenging due to the requirement for manual model design and configuration.

One potential solution to address the limited capacity of a model is to provide it with more capacity than necessary and enable every layer to contribute to the decision-making process for each task, thereby creating an ensemble model. An example of such an approach is the Incremental Adaptive Deep Model (IADM), which leverages the outputs from these layers as input to a shallow network. This shallow network employs an attention mechanism to make the final prediction (?).

Figure 12 provides an overview of the Hard Sub-networks category. Most studies within this category focus on exploring the optimal set of parameters to effectively solve multiple tasks. It is crucial to note that the ideal parameter set should also be minimal due to resource limitations. As the timeline progresses, all available resources (model parameters) become occupied, thereby rendering the model incapable of learning any additional tasks without experiencing CF. In an attempt to address this issue, some researchers have introduced a new category of strategies known as Dynamic Networks, which allow for model expansion when required. The dynamic networks category will be discussed in the upcoming section.

### 3.4 Dynamic Networks

Dynamic Network category suggests that the CF problem is caused by the overlap of knowledge within neural network connections, but beyond this, it also suggests that a fixed number of resources (parameters) will not allow learning new tasks in perpetuity without forgetting previous ones. Therefore, the Dynamic Network techniques circumvent this problem, allowing the model to expand its capacity when necessary. The pioneering work in this idea was an analytical system based on the pseudoinverse of the input matrix, introduced by ? (?), which was used to identify the optimal parameters after adding a new neuron on the output layer. While this approach may effectively solve the problem of CF, it is limited to networks that follow the Extreme Learning Machine (ELM) (?) architecture. ELMs are neural networks that consist of an input layer, a hidden layer, and an output layer. Because this approach relies on the specific structure of ELMs, it cannot be applied to DNN architectures that have more complex network structures.

In the context of DNN architectures, ? (?) start an investigation on this category by proposing Progressive Neural Networks (PNN). PNN replicates all the network’s structure for each new task, initializing the weights of the new network with the most recently trained ones. In addition, each layer l k subscript 𝑙 𝑘 l_{k} italic_l start_POSTSUBSCRIPT italic_k end_POSTSUBSCRIPT of task t i subscript 𝑡 𝑖 t_{i} italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT is connected to the input of layer l k + 1 subscript 𝑙 𝑘 1 l_{k+1} italic_l start_POSTSUBSCRIPT italic_k + 1 end_POSTSUBSCRIPT in task t i + 1 subscript 𝑡 𝑖 1 t_{i+1} italic_t start_POSTSUBSCRIPT italic_i + 1 end_POSTSUBSCRIPT . PNN shows that expanding the capacity of the model is a possible way to handle CF in DNNs. The Expert Gate (?) method uses one model for each task; however, each model is equipped with an auto-encoder (AE) and is considered an expert to solve its task and has its parameters frozen after the training session. It is better than PNN to learn a new task because when a new model is required, the expert with better results on current training data is cloned. Therefore, it is not limited to the last model. Moreover, the AEs address the issue of identifying the task to which a specific sample belongs.

Another work that creates a new DNN model for each task is the Dual-Memory (?). For each task, a new DNN is trained on data of the current task to produce an embedding. This embedding is concatenated with embeddings from all previous DNNs trained. This concatenation creates a rich representation of the data across all tasks learned so far, allowing the model to retain knowledge from previous tasks. Next, a shared shallow neural network selects a random subset of this concatenation to predict the class of a sample. The weights of a novel DNN are always initialized with the trained weights of the last DNN learned and frozen when the training session finishes. The main problem with these works is the inefficient use of resources, because they create a new entire DNN for each new task, even though the learning of the new task is faster as it takes advantage of the knowledge acquired by the old networks. Therefore, it required more research to minimize the increase of resources, looking for modular expansion, where growth is sub-linear (?).

Despite the inherent resource-intensive nature of Transformers architectures, certain researchers employ the Dynamic Networks approach to achieve enhanced accuracy and circumvent the challenge of CF in extended task sequences (?, ?). Leveraging the representational capabilities offered by Transformers, ? (?) propose to use a shared Transformer encoder (Vision in Transformer) to generate tokens from samples and increase the structure of the decoder to classify each task. More recently, expert modules to adapt the stimulus in internal layers of Transformers are created for each task and freeze them. In order to achieve a sub-linear growth rate, an effective strategy involves sharing modules across multiple tasks, as emphasized by ? (?). It is crucial to acknowledge that the Transformer model under consideration is a pretrained model, whereby solely the modules undergo incremental learning during the process.

In order to mitigate the escalating rate of resource consumption, ? (?) adopt a contrasting approach and suggest the utilization of a singular DNN for feature extraction, accompanied by multiple shallow networks, each dedicated to a specific task. By employing a shared feature extractor across all tasks, they propose the adoption of a pretrained model or the freezing of the feature extractor after training the initial task. During a training session, only the shallow network specialized in the current task undergoes updates. The inference is performed by aggregating the outputs of all classifiers, employing an averaging technique. The authors advocate that this approach proves effective due to its functioning as a Boosting classifier, ensuring improved classification performance while utilizing resources more efficiently.

? (?) propose to add only a new softmax layer in the model for each new task. This method is known as Learn without Forgetting (LwF). This method requires a specific training algorithm. Before starting a new training session, the model creates a set of soft labels for each data. A soft label is the probability distribution over multiple possible classes, in this case, it is generated by the softmax layers of the model. The soft label helps the model to generalize and is used to apply KD (?) in the training session. When the layers generate the labels, they are known as Teachers. On the other hand, when the layers use these labels to acquire knowledge, they are known as Students. In the training session, it applies KD in those softmax layers while using the current task data to train the new softmax created.

The method described previously utilizes a shared feature extractor to generate an embedding representation prior to the softmax layer. However, one significant drawback of this approach is that the embedding can be altered as the network learns new tasks. As a result, it becomes difficult to maintain the original knowledge, since the embedding may undergo substantial modifications during training. This presents a challenge for developing effective algorithms to avoid CF following that methods (?).

Other works focused on research metrics to indicate where and/or when the model has to increase. ? (?) proposed a metric that measures how well a new task is fitted by filters in each layer of a CNN already trained. This metric is called Averaged Response Variance (ARV) and is calculated as ARV = 1 / N ⁢ ∑ n = 1 N R n ARV 1 𝑁 superscript subscript 𝑛 1 𝑁 subscript 𝑅 𝑛 \mathrm{ARV}=\nicefrac{{1}}{{N}}\sum_{n=1}^{N}R_{n} roman_ARV = / start_ARG 1 end_ARG start_ARG italic_N end_ARG ∑ start_POSTSUBSCRIPT italic_n = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_N end_POSTSUPERSCRIPT italic_R start_POSTSUBSCRIPT italic_n end_POSTSUBSCRIPT , where N 𝑁 N italic_N is the number of filters in each layer and R 𝑅 R italic_R is the variance of the filter outputs after processing the data set of the new task. They proposed to change the model’s structure based on this metric. When the ARV value is large, they augment the number of filters to fit the new task knowledge. ? (?) show that other metrics can be used to take this decision. They proposed using overfitting and underfitting metrics. to modify the number of neurons in a layer and the number of layers if necessary. Although the accuracy results are comparable across different approaches, the growth of the model may vary depending on the task and its order. Thus, it is crucial to evaluate each scenario individually to determine which approach has the least growth rate.

Rather than relying on a metric, some researchers have opted for a trainable method where an auxiliary entity makes the decision. Neural Architecture Search (NAS) was used to expand a model until discovering the optimal structure to support the incoming task (?). They consider three operations to find the optimal structure: i) add new filters to the layer, freezing the old ones; ii) reuse the old frozen ones; and, iii) create a new layer from scratch. In parallel, (?) trained an agent with RL, in an end-to-end fashion, to decide when and where to add nodes on a neural network to support new tasks. Those methods generally spend more time to reach a final version of the model, however, can find better structures (lower size). Hierarchical methods can also be used to decide when expanding the model. Tree-CNN (?) uses a tree structure to define its model. Each node can give a prediction or take it to another node to decide. Each node has its own CNN to predict. In addition, a child node is created when a new task is presented, and the node cannot distinguish the new task from any other that it already learned. The main problem with this method is that when it makes a mistake at some intermediate level, the final inference will also be wrong.

A different research direction within this category involves the reuse of a “frozen” model and replication of only a specific portion of the original architecture. Unlike other approaches, this technique does not require any modifications to the overall structure of the DNN. Instead, the updated weights for only the relevant part of the model are retained, while the remaining weights remain unchanged. This approach can provide an efficient means of preserving the original knowledge in the network while allowing for the acquisition of new information for specific tasks. ? (?) creates a new DNN with a structure identical to that of the pretrained one, however, with the weights initialized randomly. Thus, in a training session, the model can choose whether to use a l k subscript 𝑙 𝑘 l_{k} italic_l start_POSTSUBSCRIPT italic_k end_POSTSUBSCRIPT layer from the pretrained DNN or from the recently created one. Therefore, only a part of the DNN needs to be stored and the technique can be applied to any architecture. The results show that the initial layers are generally taken from the pretrained DNN because it contains the base features. However, they do not present tests in a sequence of tasks with drift domain.

Based on the results of ? (?), which show that the initial layers of a CNN learn low-level characteristics, ? (?) proposed SeNA-CNN. This is a special case of CNN that has five convolutional layers. When the model needs to learn a new task, it just replicates layers three onwards for each new task and shares the first two. Different from PNN there are no connections between layers of different tasks. Both PNN and SeNA-CNN do not know what task a sample belongs to. To handle this issue in SeNA-CNN, ? (?) published an improvement, where another CNN equipped with a softmax output layer decides which branch to use in prediction time.