Measuring Catastrophic Forgetting in Neural Networks — part 3/4
Cleaned-text code points [31579, 47227). Bibliography omitted here; source.md includes it.
PathNet performed best overall on the data permutation experiments, with the exception of CUB-200. However, PathNet requires being told which session each test instance is from, whereas the other models do not use this information. This may give it an unfair advantage. PathNet works by locking the optimal path for a given session. Because permuting the data does not reduce feature overlap, the model requires more trainable weights (less feature sharing) to build a discriminative model, causing PathNet to saturate (freeze all weights) more quickly. When PathNet reaches the saturation point, the only trainable parameters are in the output layer. While EWC was the second best performing method in the permutation experiments, it only redirects plasticity instead of freezing trainable weights.
Both GeppNet variants performed best at incremental class learning. These models make slow, gradual changes to the network that are inspired by memory consolidation during sleep. For these models, the SOM-layer was fixed to 23 × 23 23\times 23 to have the same number of trainable parameters as the other models. With 100-200 classes, this corresponds to 2-5 hidden layer neurons per class respectively. The experiments on MNIST in ? ( ? ) used 90 hidden-layer neurons per class, so their performance may improve if their model capacity was significantly increased, but this would demand more memory and computation.
EWC performed best on the multi-modal experiment. This may be because features between the two modalities are non-redundant. We hypothesize that EWC is a better choice for separating non-redundant data and PathNet may work well when working with data that has different, but not entirely dissimilar, representations. To explore this, we used the Fast Correlation Based Filter proposed by ? ( ? ) to show the features in MNIST and AudioSet are more redundant than those in CUB-200 (see supplemental material). The performance of EWC and PathNet for both the data permutation and multi-modal experiments are consistent with this hypothesis.
Table 2 shows the memory constraints and usage of each model. While we kept the number of trainable parameters roughly the same across all models in their hidden layers, some require additional memory resources. PathNet generates a new output layer for each session. Both GeppNet variants store all training data and rehearse over it during their incremental learning stage. The creators of EWC stored validation data from all previous sessions and used it to minimize forgetting when learning a new session. This was not done in our experiments to fairly compare it to the other models, which only had access to validation data for the current session.
Table 4 shows the total time to train each model for the data permutation and incremental class learning experiments using CUB-200. Both variants of GeppNet are orders of magnitude slower because they train the model one sample at a time. PathNet is also very slow at the data permutation task because the optimal path through a large DCNN needs to be found for each permutation. The fixed-size models are noticeably faster; however, only EWC was effective at mitigating catastrophic forgetting (in the data permutation and multi-modal experiments).
In general, models that expand as a function of the number of sessions and those that are allowed to store data from prior sessions may have limited real-world application. In our opinion, methods for mitigating catastrophic forgetting should have the amount of total memory they use constrained. While our summary statistics did not take this into account, it is an important factor in deploying a method that learns incrementally. This is the reason we chose to keep the number of trainable parameters fixed across all models.
An alternative would have been to tune the number of trainable parameters in each model for each experiment, which is what we did for the data permutation and incremental class learning experiments as well (see Supplemental Materials for details). Although in most cases the base performance increased, there were no changes to any of our conclusions on which model/mechanism yielded superior performance. The one interesting thing we did observe is that the sparsity model (i.e. FEL) can sometimes improve significantly; however, the cost is a 40x increase in the model’s memory footprint. This reinforces our claim that a model that only uses the sparsity mechanism to mitigate catastrophic forgetting may not be ideal in a deployed environment. We urge future incremental learning algorithm creators to take memory footprint into account, especially when comparing to other models.
Our metrics could be expanded to other training paradigms such as reinforcement learning, unsupervised learning, etc. In reinforcement learning, the agent learns an initial study-session (e.g. an ATARI game), which represents the base knowledge. We would track the performance of the base-knowledge as the model learns additional games and ensure that the model is learning new games as well. The main difference is that the performance metrics would be normalized by the maximum performance for each study-session when the model only has to learn that single study session. In unsupervised learning, we could follow the experiments performed by [ 2016 ] where the metrics would be the same, but we would train the models using a different loss function (e.g. reconstruction error).
## Conclusion
In this paper, we developed new metrics for evaluating catastrophic forgetting. We identified five families of mechanisms for mitigating catastrophic forgetting in DNNs. We found that performance on MNIST was significantly better than on the larger datasets we used. Using our new metrics, experimental results (summarized in Table 5 ) show that 1) a combination of rehearsal/pseudo-rehearsal and dual-memory systems are optimal for learning new classes incrementally, and 2) regularization and ensembling are best at separating multiple dissimilar sessions in a common DNN framework. Although the rehearsal system performed reasonably well, it required retaining all training data for replay. This type of system may not be scalable for a real-world lifelong learning system; however, it does indicate that models that use pseudorehearsal could be a viable option for real-time incremental learning systems. Future work on lifelong learning frameworks should involve combinations of these mechanisms. While some models perform better than others in different scenarios, our work shows that catastrophic forgetting is not solved by any single method. This is because there is no model that is capable of assimilating new information while simultaneously and efficiently preserving the old. We urge the community to use larger datasets in future work.
## Acknowledgements
A. Abitino was supported by NSF Research Experiences for Undergraduates (REU) award #1359361 to R. Dube. We thank NVIDIA for the generous donation of a Titan X GPU.
## Appendix A Supplemental Material
### Training Parameters
### Reproduction Validation Experiments
In the following section we have documented our results from our reproduction of some of the experiments previously done with each of these models. PathNet is not included in this section because we were able to get the model directly from ? ( ? ).
Fig. S1 demonstrates the results for our implementation of EWC from the MNIST experiment proposed by ? ( ? ). Unlike the training methodology employed in our main paper, for the reproduction, we used the validation data from previous permutations to help retain previously trained tasks which is consistent with the original implementation of EWC. The results show the mean test accuracy across all permuted datasets seen so far. We performed a grid search across the hyperparameters (hidden layer size and learning rate) listed in the paper. Our model performs similarly to the one in ? ( ? ).
Table S6 contains results from our GeppNet and GeppNet+STM model verification experiment. Each test “Inc-X” involves training the base with every class except for “X” and then adding Class “X” incrementally. ? ( ? ) do not list specific percentages for each test, but the results in Table S6 are similar to the author’s. The three reported metrics, in order, include the accuracy of the base prior to the incremental training step, the accuracy of the new class after the incremental training step, and the overall accuracy of all test data after the incremental training step.
Table S7 shows the results from the FEL verification experiment. We reproduced the non-stationary MNIST classification task with all ten digits proposed by ? ( ? ). Complete reproducibility was difficult because the authors do not state the learning rate or number of epochs for training, but the results are still comparable.
### Plots and Tables for Experimental Results
In this section we provide plots and tables demonstrating the performance of the various models on the data permutation, incremental class learning, and multi-modal experiments. Additionally, we provide a comparison of results on the MNIST dataset to results on the harder CUB-200 and AudioSet datasets.
#### Data Permutation Experiment
Fig. S2 shows the results of the data permutation experiment on the MNIST, CUB-200, and AudioSet datasets. The first column of Fig. S2 shows the performance of the first session (original data) as the network learns new permutations and the second column of Fig. S2 shows the performance of the current permutation to demonstrate that the network is still learning new information. Although GeppNet and GeppNet+STM appear to be retaining the original task, they do not appear to be acquiring new information.
While performance is worse for all models on the CUB-200 and AudioSet datasets than on MNIST (See Fig. S2 and Table 2 ), some models exhibit similar trends in behavior independent of the dataset. In particular, GeppNet and GeppNet+STM retain the original data, but are unable to learn new information for both the CUB-200 and AudioSet datasets, which is similar to the behavior they exhibited on MNIST. In addition, FEL is prone to catastrophically forgetting the original data while maintaining the ability to learn new information, with worse performance than the MLP for learning new information on all three datasets.
While EWC and PathNet have the best overall performance, both models perform worse on the CUB-200 and AudioSet datasets than on MNIST. Although PathNet is able to retain some knowledge of the original data while still maintaining the ability to learn new information on the CUB-200 and AudioSet datasets, its retention accuracy and newly trained task accuracy are much lower than in the MNIST experiments. Additionally, the EWC and MLP models exhibit similar behavior to one another on AudioSet with both models catastrophically forgetting the original data, while still maintaining some ability to learn new information.
For the permutation task on the CUB-200 dataset, EWC performs the best, with similar trends to its performance on MNIST. With many of the models yielding significantly different trends and worse overall performance for the permutation task on the CUB-200 and AudioSet, it is important to consider scalability to large datasets before choosing a model for an incremental learning based task.
#### Incremental Class Learning Experiment
Fig. S3 shows the results of the incremental class learning experiment. Similar to the permutation task, results for the incremental class learning experiment (See Fig. S3 and Table 2 ) on the CUB-200 and AudioSet are much worse than on MNIST. Overall, MLP and EWC do not perform well for the incremental task and GeppNet, GeppNet+STM, and FEL perform the best on all three datasets, with significantly better results on the MNIST dataset. Both GeppNet and GeppNet+STM are capable of retaining prior knowledge while also learning new classes; however, GeppNet performs better since it trains for every iteration (instead of only during the sleep phase).
#### Multi-Modal Experiment
Table S8 shows the results for the multi-modal experiment. The results indicate that EWC performs the best for this task, which is consistent with the results presented in Table 2 .
### Fast Correlation Based Filter
In this paper, we used the Fast Correlation Based Filter (FCBF) to measure feature redundancy in each dataset [ 2003 ] . FCBF uses symmetric uncertainty to measure the independence (inverse redundancy) between two random variables X , Y X,Y . Symmetric uncertainty is defined in Eq. 5 where H ( X ) H\left(X\right) is the entropy of X X (Eq. 6 ), H ( X | Y ) H\left(X|Y\right) is the entropy of X X after observing Y Y (Eq. 7 ), and I G ( X | Y ) IG\left(X|Y\right) is the information gain between X X and Y Y (Eq. 8 ).
S U ( X , Y ) = 2 ⋅ I G ( X | Y ) H ( X ) + H ( Y ) SU\left(X,Y\right)=2\cdot\frac{IG\left(X|Y\right)}{H\left(X\right)+H\left(Y\right)} (5)
H ( X ) = − ∑ i P ( x i ) l o g 2 ( P ( x i ) ) H\left(X\right)=-\sum_{i}P\left(x_{i}\right)log_{2}\left(P\left(x_{i}\right)\right) (6)
H ( X | Y ) = − ∑ j P ( y j ) ∑ i P ( x i | y j ) l o g 2 ( P ( x i | y j ) ) H\left(X|Y\right)=-\sum_{j}P\left(y_{j}\right)\sum_{i}P\left(x_{i}|y_{j}\right)log_{2}\left(P\left(x_{i}|y_{j}\right)\right) (7)
I G ( X | Y ) = H ( X ) − H ( X | Y ) IG\left(X|Y\right)=H\left(X\right)-H\left(X|Y\right) (8)
Table S9 shows the total number of non-redundant features for each dataset along with the percentage of features that are not redundant in each dataset. The results show that the features in MNIST and AudioSet are noticeably more redundant than the features found in CUB-200.
Figure S4 is a visualization of these results, where we show the symmetric uncertainty matrix for each dataset. The results are a F × F F\times F matrix where F F is the dimensionality of the feature vector (e.g. CUB-200 is 2048). The bright areas represent features that are strongly correlated with one another (they are more redundant). The results show significant feature overlap in MNIST which is expected since it is a gray-scale image with zero values in the background. AudioSet also has sub-diagonals across the matrix which correspond to highly correlated features that repeat over some interval. Each AudioSet sample consists of ten 128-dimensional sub-vectors that are concatenated together to form a single vector. Each sub-vector is the feature representation of the audio signal for a single second. Since the sounds repeat across the entire ten seconds, the corresponding features are strongly correlated in those locations. The CUB-200 features appear to not be strongly correlated. This is probably because each sample is the ResNet-50 feature representation which is highly discriminative.
### Ideal Model
Table S10 show the experimental results when the model capacity is not constrained; that is, we performed a hyperparameter search to find the best model for each model/dataset combination. The base results are a bit higher than the results where we constrained the model capacity (Table 2 ), but the main conclusions remain the same.
## Instructions for reporting errors
We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:
Click the "Report Issue" ( ) button, located in the page header.
Tip: You can select the relevant text first, to include it in your report.