In a continual learning setting, new categories may be introduced over time, and an ideal learning system should perform well on both the original categories and the new categories. While deep neural nets have achieved resounding success in the classical setting, they are known to forget about knowledge acquired in prior episodes of learning if the examples encountered in the current episode of learning are drastically different from those encountered in prior episodes. This makes deep neural nets ill-suited to continual learning. In this paper, we propose a new model that can both leverage the expressive power of deep neural nets and is resilient to forgetting when new categories are introduced. We demonstrate an improvement in terms of accuracy on original classes compared to a vanilla deep neural net. Despite the multitude of successes of deep neural networks in recent years, one of the major unsolved issues is their difficulty in adapting to new tasks while retaining knowledge acquired from old ones. This problem is what continual learning aims to tackle. To successfully develop a continual learning method, one major limitation that must be overcome is the problem of catastrophic forgetting (McCloskey & Cohen, 1989) , which describes the phenomenon that a machine learning model finetuned for a new task performs poorly on the old task it was originally trained on. Various continual learning methods make different assumptions about whether the task identities and boundaries are known at training time and test time (van de Ven & Tolias, 2019; Hsu et al., 2018; Zeno et al., 2018) . In the most restrictive setting, task identities are known at both training and test time and the method is only asked to classify among the possible classes within the given task, a setting known as task learning. In domain learning, the assumption that the task identity is known at test time is removed (with all others in place) and each task is assumed to have the same number of classes with similar semantics; in this case, the method is asked to classify among the possible classes within any given task. Class learning differs from domain learning in that each task may contain a different number of classes that may have non-overlapping semantics with the classes in other tasks, and the method is asked to classify among the possible classes across all tasks. Discrete task-agnostic learning further generalizes this by removing the assumption of known task identity at training time and replacing it with an assumption of known boundaries between different tasks at training time. In the most general setting, task boundaries between different tasks are also unknown, even at training time. This setting is known as continuous task-agnostic learning, which is the setting we focus on in this paper. Note that the same method can yield very different performance under different settings; in particular, some methods may work very well on more restrictive settings, but undergo a significant performance degradation under less restrictive settings. This is because different underlying strategies employed by various methods may be especially dependent on certain assumptions. As a result, care must be taken when interpreting results across different papers to ensure that evaluation is conducted under the same setting. Refer to (van de Ven & Tolias, 2019; Hsu et al., 2018) for an extensive study and discussion on the performance of various methods under different settings. Existing continual learning methods can be divided into two broad categories: those that bias the parameters towards parameter values learned on old tasks, and those that expands the model size to accommodate new tasks. In this paper, we propose a new approach that is orthogonal to these categories. Our approach neither penalizes parameter changes nor expands model size. Our key intuition is that neural nets forget because parameters at all layers need to change to adapt to new tasks. Therefore, one way to improve retention is to keep parameter changes localized. Because upper-layer parameters depend on lower-layer parameters, upper-layer parameters must change when lower-layer parameters change; hence, lower-layer parameters must be kept relatively static to prevent parameter changes throughout the network. One way to achieve this would be to explicitly regularize parameter changes in the lower layers; this is less than ideal, however, because it would reduce the network's expressive power and therefore its capability to learn on new tasks. How do we achieve this without compromising on expressive power? To arrive at a solution to this problem, we need to consider the underlying cause of parameter changes at all layers when fine-tuning on new tasks. In a neural net, each layer on its own has quite limited expressive power and is simply a linear model. As a result, if the features computed by the penultimate layer on training examples for the new task are not linearly separable, then the parameters in the layers up to the penultimate layer must change to successfully learn the new task. To avoid this, we can make the last layer more expressive and replace it with a nonlinear classifier. To this end, we propose replacing the softmax activation layer with a k-nearest neighbour classifier. We will show that this simple modification is surprisingly effective at reducing catastrophic forgetting. As discussed in Section 4, we achieve consistent results across four different datasets/splits. These results suggest that the method is successful at addressing the catastrophic forgetting problem, and hint at a possible general strategy for reducing catastrophic forgetting, namely the idea of replacing the last layer of a neural net with a non-linear classifier. In the future, we would like to explore combining the proposed approach with other orthogonal approaches for tackling catastrophic forgetting. We also plan to explore applications that can benefit from continual learning approaches, such as reinforcement learning.
We show metric learning can help reduce catastrophic forgetting This paper applies metric learning to reduce catastrophic forgetting on neural networks by improving the expressiveness of the final layer, leading to better results in continual learning.