Measuring Catastrophic Forgetting in Neural Networks — part 2/4

Cleaned-text code points [15928, 31579). Bibliography omitted here; source.md includes it.

Caltech-UCSD Birds-200 (CUB-200) is an image classification dataset containing 200 different bird species [ 2011 ] . We use the 2011 version. Each high-resolution image is turned into a 2048-dimensional vector with ResNet-50 [ 2016 ] , which is a deep convolutional neural network (DCNN) pre-trained on ImageNet [ 2015 ] . Extracting image features from the last hidden (fully-connected) layer of pre-trained DCNNs is a common practice in computer vision. We report mean-per-class accuracy, which is the CUB-200 standard.

#### AudioSet

AudioSet [ 2017 ] is a hierarchically organized audio classification dataset built from YouTube videos. It has over 2 million human-labeled, 10 second sound bytes drawn from one or more of 632 classes. We used the pre-extracted frame-wise features from AudioSet concatenated in order. These features were extracted using a variant ResNet-50 for audio data [ 2017 ] , which was pre-trained on an early version of the YouTube-8m dataset [ 2016 ] . We used 100 classes from AudioSet, none of which were super or sub-classes of each other. The classes did not have any restrictions based on the AudioSet ontology, and all of them had a quality estimation of over 70%. Each audio sample can have multiple labels, so we chose training and testing samples that were labeled with only 1 of the 100 classes.

### Models Evaluated

We evaluated five models that correspond to each of the five mechanisms described in the previous section: 1) EWC, 2) PathNet, 3) GeppNet, 4) GeppNet+STM, and 5) FEL. To choose the number of parameters to use across models, we established a baseline MLP architecture that performed well for CUB-200 and AudioSet when trained offline. The goal is to determine which mechanism(s) work best for various incremental learning paradigms. To provide a fair comparison, the number of parameters in each model were chosen to be as close as possible to the number of parameters in the baseline MLP. We optimized each model’s hyperparameters to work well for our benchmarks, which are given in supplemental materials 1 1 1 Supplemental materials provided at the end of our arXiv submission: https://arxiv.org/abs/1708.02072 . The supplemental materials provides the stopping criteria for each model as defined by their creators, which involved 1) training for a fixed period of time or 2) using test accuracy to stop training early.

#### Standard Multi-Layer Perceptron

For our baseline, we use a standard MLP. Its architecture was chosen by optimizing performance using the entire training set for both CUB-200 and AudioSet, i.e., it was trained offline. The offline MLP achieves 62.1% accuracy on the CUB-200 test set and 46.1% on the AudioSet test set. We did a hyperparameter search for the number of units per hidden layer (32-4,096), number of hidden layers (2-3), and weight decay parameter (0, 10 − 4 10^{-4} , 5 ⋅ 10 − 4 5\cdot 10^{-4} ). The MLP model was also trained incrementally to measure the severity of catastrophic forgetting.

#### Elastic Weight Consolidation

EWC adds an additional constraint to the loss function L ⁡ ( θ ) L\left(\theta\right) , i.e., L ⁡ ( θ ) = L t ​ ( θ ) + ∑ i λ 2 ​ F i ​ ( θ i − θ A , i ∗ ) 2 , L\left(\theta\right)=L_{t}\left(\theta\right)+\sum\limits_{i}\frac{\lambda}{2}F_{i}\left(\theta_{i}-\theta_{A,i}^{*}\right)^{2}, (1) where L ⁡ ( θ ) L\left(\theta\right) is the combined loss function, θ \theta is the network’s parameters, L t ​ ( θ ) L_{t}\left(\theta\right) is the loss for session B t B_{t} , λ \lambda is a hyperparameter that indicates how important the old task(s) are compared to the new task, F F is the Fisher information matrix, and θ A ∗ \theta_{A}^{*} are the trainable parameters (weights and biases) important to previously trained tasks. The Fisher matrix is used to constrain the weights important to previously learned tasks to their original value; that is, plasticity is directed to the trainable parameters that contribute the least to performing previously trained tasks. The size of the hidden-layer was chosen to match the baseline MLP’s capacity.

#### PathNet

PathNet is a fixed size neural network that uses a genetic algorithm to find the optimal path through the network. Only this path is trainable when learning a particular session, which is why the authors described their model as an evolutionary dropout network. PathNet creates an independent output layer for each task in order to preserve previously trained tasks, and it cannot be used without modifications for incremental class learning. Since entire portions of the network are sequentially frozen as new tasks are learned, there is a risk of PathNet losing its ability to learn once the maximum capacity is reached. PathNet’s capacity was chosen to match the capacity of the MLP baseline.

#### GeppNet

GeppNet and GeppNet+STM are biologically-inspired approaches that use rehearsal to mitigate forgetting. In these models, training the initial task starts by initializing the SOM-layer, which is used to project the probability density of the input to a higher two-dimensional lattice. The SOM-layer features are passed to a linear regression classification layer to make a prediction. During training, the SOM-layer is initialized with the first session for a fixed-period of time, and then the SOM- and classification-layers are trained jointly. The SOM-layer is only updated when a training example is determined by the model to be novel (i.e., using the prediction probabilities to generate a confidence measure). After GeppNet has been trained on the initial session for a fixed period of time, it incrementally learns subsequent sessions. GeppNet performs updates to the SOM-layer and classification-layer when a training example is considered novel. When GeppNet+STM detects novelty, it instead uses a fixed-size short-term memory buffer to store that training example, which then replays it during a sleep phase. The sleep phase repeats after a fixed number of training iterations. Since the replay queue has a fixed-size (i.e., older examples are replaced), the GeppNet+STM model will train more efficiently than GeppNet. GeppNet stores all previous training data and replays it along with the previous data during a portion of its incremental learning step. GeppNet+STM also stores all previous and new training data; however, each training example is only replayed if the model is uncertain on the prediction. In addition, GeppNet+STM is capable of making real-time predictions by determining if the desired memory is in short-term memory (the memory buffer) or in long-term storage (the SOM- and classification-layers).

#### Fixed Expansion Layer

FEL uses sparsity to mitigate catastrophic forgetting [ 2013 ] . FEL is a two hidden-layer MLP where the second hidden-layer (FEL-layer) has a higher capacity than the first fully-connected layer, but the weights are sparse and remain fixed through training. Each FEL-layer unit is only connected to a subset of the units in the first hidden layer, and these connections are split between excitatory and inhibitory weights. Only a subset of the FEL-layer units are allowed to have non-zero output to the final classification layer, which causes only some of the units in the first hidden layer to be updated.

## Experiments and Results

We have established three benchmark experiments for measuring catastrophic forgetting: 1. Data Permutation Experiment - The elements of every feature vector are randomly permuted, with the permutation held constant within a session, but varying across sessions. The model is evaluated on its ability to recall data learned in prior study sessions. Each session contains the same number of examples.

2. Incremental Class Learning - After learning the base set, each new session learned contains only a single class.

3. Multi-Modal Learning - The model incrementally learns different datasets, e.g., learn image classification and then audio classification.

For the data permutation and incremental class learning experiments, each model was also evaluated on MNIST. The goal is to examine whether results on MNIST generalize to the real-world datasets. More results, including detailed plots, can be found in the supplementary materials.

### Evaluation Metrics

We propose three new metrics to evaluate a model’s ability to retain prior sessions while still learning new knowledge,

Ω b ​ a ​ s ​ e = 1 T − 1 ​ ∑ i = 2 T α b ​ a ​ s ​ e , i α i ​ d ​ e ​ a ​ l \Omega_{base}=\frac{1}{T-1}\sum_{i=2}^{T}\frac{\alpha_{base,i}}{\alpha_{ideal}} (2) Ω n ​ e ​ w = 1 T − 1 ​ ∑ i = 2 T α n ​ e ​ w , i \Omega_{new}=\frac{1}{T-1}\sum_{i=2}^{T}\alpha_{new,i} (3)

Ω a ​ l ​ l = 1 T − 1 ​ ∑ i = 2 T α a ​ l ​ l , i α i ​ d ​ e ​ a ​ l \Omega_{all}=\frac{1}{T-1}\sum_{i=2}^{T}\frac{\alpha_{all,i}}{\alpha_{ideal}} (4) where T T is the total number of sessions, α n ​ e ​ w , i \alpha_{new,i} is the test accuracy for session i i immediately after it is learned, α b ​ a ​ s ​ e , i \alpha_{base,i} is the test accuracy on the first session (base set) after i i new sessions have been learned, α a ​ l ​ l , i \alpha_{all,i} is the test accuracy of all of the test data for the classes seen to this point, and α i ​ d ​ e ​ a ​ l \alpha_{ideal} is the offline MLP accuracy on the base set, which we assume is the ideal performance. Ω b ​ a ​ s ​ e \Omega_{base} and Ω n ​ e ​ w \Omega_{new} are normalized area under the curve metrics. Ω b ​ a ​ s ​ e \Omega_{base} measures a model’s retention of the first session, after learning in later study sessions. Ω n ​ e ​ w \Omega_{new} measures the model’s ability to immediately recall new tasks. By normalizing Ω b ​ a ​ s ​ e \Omega_{base} and Ω a ​ l ​ l \Omega_{all} by α i ​ d ​ e ​ a ​ l \alpha_{ideal} , the results will be easier to compare between datasets. Unless a model exceeds α i ​ d ​ e ​ a ​ l \alpha_{ideal} , results will be between [ 0 , 1 ] \left[0,1\right] , which enables comparison between datasets. Ω a ​ l ​ l \Omega_{all} computes how well a model both retains prior knowledge and acquires new information.

### Experimental Results

#### Data Permutation Experiment

This experiment evaluates a model’s ability to retain multiple representations of the dataset, with each representation learned sequentially. These representations are created by randomly permuting the elements of the input feature vectors, with the random permutation changing between sessions. An identically permuted test set is used along with each session. This paradigm provides overlapping tasks in which each session contains the same information and categories, so each session is of equal complexity. This paradigm is identical to that used by ? ( ? ) and ? ( ? ).

Results are given in Table 2 . In nearly every case, Ω a ​ l ​ l \Omega_{all} is greater for MNIST than on CUB-200 or AudioSet, demonstrating the need for alternative incremental learning benchmarks. To some extent, EWC, PathNet, GeppNet, and GeppNet+STM retain prior knowledge without forgetting; however, GeppNet and GeppNet+STM fail to learn new sessions. PathNet and EWC seem to retain base knowledge while still learning new information; however, PathNet performs better on AudioSet and MNIST, while EWC performs better on CUB-200 (see discussion).

#### Incremental Class Learning

In the incremental class learning experiment, a model’s ability to sequentially learn new classes is tested. The first session learned contains training data from half of the classes in each dataset: 5 for MNIST, 100 for CUB-200, and 50 for AudioSet. Once this base set was learned, each subsequent session contained training data from a single new class. We measure mean-per-class accuracy on the base set after each new class is learned to assess a model’s long-term memory. We also calculate the accuracy of each class after it is trained to ensure the model is still learning, and we calculate the performance of all previously learned classes.

PathNet is incapable of learning new classes incrementally because it creates a separate output layer for each additional session. Accessing that output layer during prediction time requires a priori information on which session the model needs to access. This means PathNet requires the testing label to make the appropriate prediction, which would result in a misleading high test accuracy. For this reason, we omitted PathNet from this experiment.

Results are summarized in Table 2 and Fig. 2 contains plots for the mean-per-class test accuracy for all classes seen so far. The only models that were able to both retain the base knowledge and learn new classes were GeppNet, GeppNet+STM, and FEL, with the clear winner being GeppNet. Much like the data permutation experiment, the CUB-200 and AudioSet results were noticeably lower than the MNIST results. GeppNet+STM did well at retaining the base set, but it struggled to learn new classes on CUB-200 and AudioSet. This could be because the model only trains during sleep for efficiency reasons. Additionally, the short-term memory buffer is emptied after training each study session, which is when the model is evaluated. This type of model could work better in a real-time environment. FEL learned new classes well, but suffered from forgetting of the base set. FEL may benefit from larger model capacity, but this would require more memory/processing power.

#### Multi-Modal Experiment

The goal of the multi-modal experiment is to determine if a network can learn and retain multiple dissimilar tasks that have 1) inputs with different dimensionality and feature distributions and 2) a different number of classes. A system like this could be useful for learning tasks that have multi-modal data using a single network and could be more efficient than building a separate neural network for each modality (e.g., video has visual and audio information). In this experiment, we evaluated each model’s ability to perform image and audio classification with CUB-200 and AudioSet respectively. In this experiment, there are only two incrementally learned sessions, where each session contains AudioSet or CUB-200. We compare learning AudioSet first then CUB-200 (AS/CUB) and learning CUB then AudioSet (CUB/AS).

The ResNet features obtained from CUB-200 have a higher dimensionality than the AudioSet features, so we zero-padded the AudioSet input to match the dimensionality of CUB-200. This experiment is done by training one dataset to completion followed by training the other dataset to completion (and vice-versa). Once both modalities have been trained, we evaluate the first modality that was trained in order to measure how well the model was able to retain what it learned about the first task.

Table 2 shows summary results for the multi-modal experiment, where the corresponding row is the modality that was trained first, i.e. the row for CUB-200 is where CUB-200 is learned first followed by AudioSet. Additional results are in supplementary materials. Although several models perform well at one of the two experiments, EWC is the only model capable of preserving the first modality while also learning the second modality for both cases, which we explore further in the discussion.

## Discussion

In our paper we introduced new metrics and benchmarks for measuring catastrophic forgetting. Our results reveal that none of the methods we tested solve catastrophic forgetting, while also enabling the learning of new information. Table 3 summarizes these results for each of our experiments by averaging Ω a ​ l ​ l \Omega_{all} over all datasets. While no method excels at incremental learning, some perform better than others.