Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy β€” part 5/9

Cleaned-text code points [62020, 77087). Bibliography omitted here; source.md includes it.

To overcome the high computational cost associated with calculating the Fisher Information Matrix (FIM) during post-processing, the Synaptic Intelligence (SI) technique offers a viable solution. SI involves the creation of a matrix Ξ© k subscript Ξ© π‘˜ \Omega_{k} roman_Ξ© start_POSTSUBSCRIPT italic_k end_POSTSUBSCRIPT that represents the importance of each parameter. This matrix is calculated during the training session by applying a small perturbation to the parameter values and observing the resulting degradation in the solution, as described by ? (?). Parameters that have a more negative impact are considered more important.

Similarly, ? (?) also employs an importance matrix within the SI framework, but with two main differences. Firstly, instead of calculating the matrix during training, they compute it during inference time. This shift in computation reduces the post-processing computational burden. Secondly, they nullify the output weights of nodes deemed unimportant. When the model needs to learn a new task, these nullified weights are randomly reinitialized. This approach provides the model with an opportunity to acquire new and different knowledge. By nullifying and reinitializing the weights, the model can adapt its architecture and synaptic connections to accommodate the requirements of the new task.

Another approach to address the computational cost during post-processing involves considering all parameters as important and striving to maintain them within the same region defined by the first task. This can be achieved through the utilization of the Monte Carlo Variational Inference method (?). By minimizing the variation of all model parameters using the Kullback-Leibler divergence, the model seeks to preserve their values across different tasks (?).

To further refine this approach, ? (?) introduced the concept of online Laplace Approximation with Kronecker. By leveraging this technique, they estimated the posterior distribution of the parameters. This estimation not only considers the correlation between a parameter and itself in the next task but also accounts for the correlation with other parameters. By incorporating these correlations, the model can capture more nuanced dependencies between parameters, resulting in a more accurate estimation of the posterior.

? (?) advocate that some things need to be forgotten for new knowledge to be learned. They maintain a matrix Ξ© Ξ© \Omega roman_Ξ© during inference time. This matrix is updated by each inference, observing which parameters are more activated when the test data is processed by the model. In this sense, the model gives more importance to connections that are more used in test time. This method was called Memory Aware Synapses (MAS). The principal problem of MAS is that it tends to forget rare classes quickly. ? (?) show that this selective forgetting is consistent with the underlying mechanism of biological forgetting. Thus, they add a term to calculate the weight importance matrix to decide which parameters, despite being important for the previous task, should be modified to maximize the probability of the new task being learned.

To incorporate a temporal factor in the strength of synapses, the Benna-Fusi model proposed by ? (?) was integrated into a DNN. This model introduces the concept of gradually changing synapse strength over time, resulting in some synapses being reinforced while others are lost. This temporal modulation of synapse strength enables the network to adapt and incorporate new information while retaining important knowledge from previous tasks. Although initially tested in the context of RL, where there is a natural notion of time due to the sequential nature of agent actions, the concept of temporal modulation can be extended to other domains as well. By incorporating the Benna-Fusi model into a DNN, researchers can introduce a temporal dimension to the learning process, allowing the network to dynamically adjust its synapse strengths based on the sequence of tasks encountered.

There are also alternative strategies for the creation of Soft Sub-networks, apart from penalizing critical weights. To eliminate the module of parameter importance, a technique called Incremental Moment Matching (IMM) proposes to train a set of models M t i subscript 𝑀 subscript 𝑑 𝑖 M_{t_{i}} italic_M start_POSTSUBSCRIPT italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT end_POSTSUBSCRIPT , all of them with the same structure. Each model is addressed to learn a unique task. In the end, all models are averaged and the resulting model should be able to solve all tasks in T 𝑇 T italic_T (?). Tasks are not learned in parallel, because for this strategy to work, the initial weights of the model M t i + 1 subscript 𝑀 subscript 𝑑 𝑖 1 M_{t_{i+1}} italic_M start_POSTSUBSCRIPT italic_t start_POSTSUBSCRIPT italic_i + 1 end_POSTSUBSCRIPT end_POSTSUBSCRIPT need to be cloned from the already trained model M t i subscript 𝑀 subscript 𝑑 𝑖 M_{t_{i}} italic_M start_POSTSUBSCRIPT italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT end_POSTSUBSCRIPT . In addition, L ⁒ 2 𝐿 2 L2 italic_L 2 regularization is applied to the weights of models M t i subscript 𝑀 subscript 𝑑 𝑖 M_{t_{i}} italic_M start_POSTSUBSCRIPT italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT end_POSTSUBSCRIPT and M t i + 1 subscript 𝑀 subscript 𝑑 𝑖 1 M_{t_{i+1}} italic_M start_POSTSUBSCRIPT italic_t start_POSTSUBSCRIPT italic_i + 1 end_POSTSUBSCRIPT end_POSTSUBSCRIPT to ensure that the parameters of the solution of the second task are close to the parameters of solution of the first task.

Another alternative to solve the selection problem on Soft Sub-networks is to try to select a subset of parameters to solve each task based on context. Generally, the input distribution for each task is regarded as the task’s context. Orthogonal Weights Modification (OWM) uses the input distribution to choose the filters used by the model to solve the tasks. It is supported by finding orthogonal filters for each task and, in inference time, using the task identifier to retrieve the right filters (?). To achieve this, when a new training session happens, the parameters can only be modified in the direction orthogonal to the subspace spanned by previous tasks. ? (?) use two RNNs to define the context: i) the main; and; ii) the auxiliary. The latter is trained to choose the optimal parameters of the former based on input distribution. The goal of the auxiliary network consists to find a subset of connections on the main network that best solves the current task without training. As a result, this subset is also used for similar tasks, and instead of forgetting happening, it improves the quality of representations. The main network is responsible to aggregate the knowledge of all tasks in their parameters. And, use the subset parameters selected by the secondary network to train in the current task with supervised learning, nullifying other parameters.

Inspired from Learn without Forgetting (LwF) - discussed in the next section -, Learn without Memorizing (LwM) uses Knowledge Distillation (KD) (?) to alleviate the forgetting in training sessions (?). While LwF is focused on Task Incremental Learning setup, the LwM is focused on Class Incremental Learning setup. This means that it does not create an entire classifier for each task, only append one output for each new class in the classifier that already exists, and regularizes the internal paths of the model to classify each new task. To achieve this, KD is applied to the last two layers of the model: the embedding layer and the classification layer. KD is a technique where a smaller, student network learns from a larger, teacher network, to improve its performance. However, in Incremental Learning, the teacher is usually the model with weights frozen in the oldest learned task.

Most of the research in this category focuses on penalizing or controlling parameter changes to increase model stability, but this can reduce plasticity, making it difficult for models to learn new tasks. While preserving prior knowledge is important, it’s essential to balance stability and adaptability to efficiently acquire new knowledge while retaining what has already been learned. Figure 10 provides an overview of this category. One challenge with these methods is that there is often a drift in the distribution of input data between tasks. If the tasks are similar, such as the MNIST splits, all methods in this group can handle the problem because the drift is low, and the samples belong to the same domain. The choice of which method to use in this group depends on training time, with Synaptic Intelligence (SI) or Incremental Moment Matching (IMM) being a better choice for shorter training times. However, when the tasks are from different domains, the drift is high, and none of these methods can handle the problem. This occurs because solving task t i + 1 subscript 𝑑 𝑖 1 t_{i+1} italic_t start_POSTSUBSCRIPT italic_i + 1 end_POSTSUBSCRIPT requires changing critical parameters that were learned to solve task t i subscript 𝑑 𝑖 t_{i} italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT . To address this problem, another research approach, known as Hard Sub-networks, was proposed. In Hard Sub-networks, the connections used to solve task t i subscript 𝑑 𝑖 t_{i} italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT are frozen, and this approach will be discussed in the next section.

#### 3.3.2 Hard Sub-networks

In the Hard Sub-networks category, the main objective is to find a set of weights inside the main model. However, different from the Soft Sub-networks category, after the set of important weights was selected for a task t i subscript 𝑑 𝑖 t_{i} italic_t start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT , they cannot be changed in the training session of tasks t i + j subscript 𝑑 𝑖 𝑗 t_{i+j} italic_t start_POSTSUBSCRIPT italic_i + italic_j end_POSTSUBSCRIPT , where j 𝑗 j italic_j represents a positive integer value. ? (?) led a pioneering study on this category proposing the Fixed Expansion Layer (FEL) Networks. FEL expands the original neural network adding a layer after each hidden layer. These additional layers are larger than their predecessor hidden layer and each neuron of the original layer is connected only to a subset of neurons in the new hidden layer. In addition, the weights of these new layers are frozen, thus they are never changed during the training phases. This expansion of the layers helps the model to avoid the overlap of knowledge over tasks. As the choice of connections between the hidden and the additional layer is random, they use an ensemble of FEL networks. Therefore, this technique is not categorized as a hybrid or dynamic because it only changes the model’s structure only at the beginning.

The selection of a subset of input neurons to combat the CF was introduced by ? (?). The proposed selection forces different regions of the input space to be learned in different nodes avoiding the knowledge overlap. ? (?) propose to use Self-Organized Map (SOM) to correlate the input with the neurons. In each training session, SOM is updated if the model fails, otherwise, just the last layer suffers fine-tuning. Therefore, for each task (represented by the statistic distribution discovered by SOM) a new subset of parameters is selected. Similar work was done by ? (?); however, instead of using SOM, they used the unsupervised algorithm KNN . To separate neurons they used Mahalanobis distance with a sample and sparse covariance matrix generated by a Shrinkage estimator (?). Other researchers concentrate on choosing parameters from the full model due to the limited resources in the input layer.

A Genetic Algorithm (GA) approach was used by ? (?) to find the best minimal subset of parameters to learn each task sequentially. When the training starts, each chromosome is a subset of parameters. The model freezes all other parameters and applies a traditional gradient learning mechanism. When the solution gets stable, the model reserves these parameters for this task. Thus, when a new task must be learned, it can only use the set of non-reserved parameters. It is important to consider solutions that use as few weights as possible, otherwise, the solution found in the first task could consume all connections.

A weight pruning introduced by ? (?, ?) shows that a model can have similar accuracy with fewer parameters if the proposed pruning-training process is incrementally executed. This process consists in nullifying some weights and retraining in the same data set until the accuracy cannot overcome a pre-defined threshold in the training. Those weights that were nullified are considered pruned and the neural network can solve the task only with the weights that are left. Therefore, ? (?) used this process to select the minimal subset of parameters for each task. They also apply L 1 subscript 𝐿 1 L_{1} italic_L start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT norm in the training phase to accelerate the pruning process. They showed that it is possible to resolve some tasks with less than 10% of total resources.

? (?) showed that with a random selection of 20 % percent 20 20\% 20 % of the weight for each task (freezing other 80 % percent 80 80\% 80 % ), the neural network can learn a sequence of 100 tasks without suffering much from CF in tasks like Permut-MNIST. For this approach to be successful, it needs to be associated with weights regularization like EWC or SI (?, ?).

PackNet uses a binary mask of weights for each task (?). This causes an overhead of storage, as the number of tasks increases. To minimize the storage needed by masks, they proposed to use of an incremental mask. That is, if the model uses parameters p 1 subscript 𝑝 1 p_{1} italic_p start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , p 2 subscript 𝑝 2 p_{2} italic_p start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT , and p 3 subscript 𝑝 3 p_{3} italic_p start_POSTSUBSCRIPT 3 end_POSTSUBSCRIPT to learn the first task, it freezes these parameters and uses them together with others (not frozen) to learn a new task. To create the mask they also used the iterative pruning method of ? (?). ? (?) extended PackNet by implementing an attention mechanism, called Hard Attention to the Task (HAT). It has a gate coupled to the output of the neurons that control the flow of information transmitted by the neuron. This mask is learned together with the task using a sigmoid function as a gate. As the sigmoid function produces a value between zero and one, they used a mechanism to force the result to be only zero or one, generating the mask. HAT was tested in eight sequential tasks, and it was able to learn them without CF of the initial one. ? (?) equip each node of the model with three more variables to control the mask. The first one is binary and defines whether or not to be adapted, while the other two define the magnitude of the adaptation. These three variables are learned via variational inference.