# Class-Incremental Exemplar Compression for Class-Incremental Learning

Zilin Luo1 Yaoyao Liu2 Bernt Schiele2 Qianru Sun1

1Singapore Management University
2Max Planck Institute for Informatics, Saarland Informatics Campus
<EMAIL_ADDRESS>{yaoyao.liu<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Exemplar-based class-incremental learning (CIL) [37] finetunes the model with
all samples of new classes but few-shot exemplars of old classes in each
incremental phase, where the “few-shot” abides by the limited memory budget.
In this paper, we break this “few-shot” limit based on a simple yet
surprisingly effective idea: compressing exemplars by downsampling non-
discriminative pixels and saving “many-shot” compressed exemplars in the
memory. Without needing any manual annotation, we achieve this compression by
generating $0$-$1$ masks on discriminative pixels from class activation maps
(CAM) [51]. We propose an adaptive mask generation model called class-
incremental masking (CIM) to explicitly resolve two difficulties of using CAM:
1) transforming the heatmaps of CAM to $0$-$1$ masks with an arbitrary
threshold leads to a trade-off between the coverage on discriminative pixels
and the quantity of exemplars, as the total memory is fixed; and 2) optimal
thresholds vary for different object classes, which is particularly obvious in
the dynamic environment of CIL. We optimize the CIM model alternatively with
the conventional CIL model through a bilevel optimization problem [41]. We
conduct extensive experiments on high-resolution CIL benchmarks including
Food-101, ImageNet-100, and ImageNet-1000, and show that using the compressed
exemplars by CIM can achieve a new state-of-the-art CIL accuracy, e.g., 4.8
percentage points higher than FOSTER [44] on 10-Phase ImageNet-1000. Our code
is available at https://github.com/xfflzl/CIM-CIL.

##  1 Introduction

Dynamic AI systems have a continual learning nature to learn new class data.
They are expected to adapt to new classes while maintaining the knowledge of
old classes, i.e., free from forgetting problems [32]. To evaluate this, the
following protocol of class-incremental learning (CIL) was proposed by Rebuffi
et al. [37]. The model training goes through a number of phases. Each phase
has new class data added and old class data discarded, and the resultant model
is evaluated on the test data of all seen classes. A straightforward way to
retain old class knowledge is keeping around a few old class exemplars in the
memory and using them to re-train the model in subsequent phases. The number
of exemplars is usually limited, e.g., $5\\!\sim\\!20$ exemplars per class
[37, 18, 50, 46, 28, 13, 26, 48, 44], as the total memory in CIL strictly
budgeted, e.g., $2k$ exemplars.

Figure 1:  The phase-wise training data in different methods. (a) iCaRL [37]
is the baseline method using full new class data and few-shot old class
exemplars. (b) Mnemonics [28] distills all training samples into few-shot
exemplars without increasing their quantity. (c) MRDC [45] compresses each
exemplar uniformly into a low-resolution image using JPEG [43]. (d) Our
approach based on the proposed class-incremental masking (CIM) downsamples
only non-discriminative pixels in the image. The legend shows the symbols of
special images generated by the methods.

This leads to a serious data imbalance between old and new classes, e.g., $20$
per old class _vs._ $1.3k$ per new class (on ImageNet-1000 [10]), as
illustrated in Figure 1a. The training is thus always dominated by new
classes, and forgetting problems occur for old classes. Liu et al. [28] tried
to mitigate this problem by parameterizing and distilling the exemplars,
without increasing the number of them (Figure 1b). Wang et al. [45] traded off
between the quality and quantity of exemplars by uniformly compressing
exemplar images with JPEG [43] (Figure 1c). As shown in Figure 1d, our
approach is also based on image compression. The idea is to downsample only
non-discriminative pixels (e.g., background) and keep discriminative pixels
(i.e., representative cues of foreground objects) as the original. In this
way, we do not sacrifice the discriminativeness of exemplars when increasing
their quantity. In particular, we aim for adaptive compression in dynamic
environments of CIL, where the intuition is later phases need to be more
conservative (i.e., less downsampling) as the model needs more visual cues to
classify the increased number of classes.

To achieve selective and adaptive compression, we need the location labels of
discriminative pixels. Without extra labeling, we automatically generate the
labels by utilizing the model’s own “attention” on discriminative features,
i.e., class activation maps (CAM) [51]. We take this method as a feasible
baseline, and based on it, we propose an adaptive version called class-
incremental masking (CIM). Specifically, for each input image (with its class
label), we use its feature maps and classifier weights (corresponding to its
class label) to compute a CAM by channel-wise multiplication, aggregation, and
normalization. Then, we apply hard thresholding to generate a $0$-$1$
mask.111Note that we do not use mask labels to do image compression because
storing them is expensive. Instead, we expand the mask to a bounding box, as
elaborated in Section 4. We notice that when generating the masks in the
dynamic environments of CIL, the optimal hyperparameters (such as the value of
hard threshold and the choice of activation functions) vary for different
classes as well as in different incremental phases. Our adaptive version CIM
tackles this by parameterizing a mask generation model and optimizing it in an
end-to-end manner across all incremental phases. In each phase, the learned
CIM model adaptively generates class- and phase-specific masks. We find that
the compressed exemplars based on these masks have stronger
representativeness, compared to using the conventional CAM.

Technically, we have two models to optimize, i.e., the CIL model and the CIM
model.222Note that the CIM model is actually a plug-in branch in the CIL
model, which is detailed in Section 4.2. These two cannot be optimized
separately as they are dependent on computation: 1) the CIM model compresses
exemplars to input into the CIL model; 2) the two models share network
parameters. We exploit a global bilevel optimization problem (BOP) [41, 7] to
alternate their training processes at two levels. This BOP goes through all
incremental training phases. In particular, for each phase, we perform a local
BOP with two steps to tune the parameters of the CIM model: 1) a temporary
model is trained with the compressed exemplars as input; and 2) a validation
loss on the uncompressed new data is computed and the gradients are back-
propagated to optimize the parameters of CIM. To evaluate CIM, we conduct
extensive experiments by plugging it in recent CIL methods,333Using “plug-in”
for evaluation is due to the fact that many baseline methods were originally
evaluated in different CIL settings. LUCIR [18], DER [48], and FOSTER [44], on
three high-resolution benchmarks, Food-101 [3], ImageNet-100 [18], and
ImageNet-1000 [10]. We find that using the compressed exemplars by CIM brings
consistent and significant improvements, e.g., $4.2\%$ and $4.8\%$ higher than
the SOTA method FOSTER [44], respectively, in the $5$-phase and $10$-phase
settings of ImageNet-1000, with a total memory budget for $5k$ exemplars.

##  2 Related Work

Class-Incremental Learning (CIL). There are three main lines of work to
address the catastrophic forgetting problem [31, 32] in CIL. Regularization-
based methods apply discrepancy (between old and new models) penalization
terms in their objective functions, e.g., by comparing output logits [24, 37],
intermediate features [18, 13, 40, 25], and prediction heatmaps [11].
Parameter-isolation-based methods increase the model parameters in each new
incremental phase, to prevent knowledge forgetting caused by parameter
overwritten. Some of them [19, 38, 47, 48, 44] proposed to progressively
expand the size of the neural network to learn new coming data. Others [22,
49, 1, 26] froze a part of network parameters (to maintain the old class
knowledge) to alleviate the problem of knowledge overwriting. Replay-based
methods assume there is a clear memory budget allowing a handful of old-class
exemplars in the memory. Exemplars can be used to re-train the model in each
new phase [37, 18, 13, 28, 46, 27, 45]. This re-training usually contains two
steps: one step of training the model on all new class data and old class
exemplars, and one step of finetuning the model with a balanced subset (i.e.,
using an equal number of samples per class) [26, 27, 18, 13, 48].

The replay-based methods focusing on _memory optimization_ [28, 45] are
closely related to our work. [28] proposed a bilevel optimization framework to
distill the current new class data into exemplars before discarding them. It
aims to improve the quality of exemplars without increasing the quantity.
Another work [45] aimed to trade-off between the quality and quantity of
exemplars by image compression using the JPEG algorithm, i.e., each exemplar
is uniformly downsampled. Ours differs from these two works in three aspects.
1) Our CIM based image compression automatically segments the discriminative
pixels in the exemplar and downsamples the non-discriminative pixels only. It
barely weakens the representativeness of the exemplar. In contrast, the
parameterization of image pixels in [28] hampers the model from capturing
high-frequency (discriminative) features from the image, especially the high-
resolution (e.g., $224\\!\times\\!224$) image [6]. 2) Our approach increases
the diversity (i.e., quantity) of old class exemplars by reducing the memory
consumption for each exemplar. In contrast, [28] keeps a fixed number of
exemplars in the memory. 3) Our approach has an adaptive image compression
strategy that fits well in the dynamic environments of CIL. In contrast, [45]
uses uniform image compression (or uniformly increasing quality parameters
from $100$ to $1$) without considering the properties of specific classes in
each learning phase.

Class Activation Map (CAM) [51] is a simple yet effective weakly-supervised
object localization method. Its model is trained with only the image-level
label and can generate pixel-level masks on foreground objects. Specifically,
the masks are the results of hard-thresholding the heatmaps produced by
feature maps and classifier weights. Advanced CAM variants include Grad-CAM
[39], ReCAM [8], AdvCAM [23], etc. Our CIM is based on the vanilla CAM because
it is computationally simple and efficient.

Bilevel Optimization Problems (BOP) [41, 7] aims to solve a nested
optimization problem, where the outer-level optimization is subjected to the
result of the inner-level optimization. It has shown effectiveness in a wide
range of machine learning areas, such as hyperparameter selection [30] and
meta-learning [14]. For tackling CIL tasks, [28] leverages BOP to
alternatively optimize the parameters of the CIL model and the parameterized
exemplars. [26] applies BOP to learn the aggregation weights of the plastic
and elastic branches in the CIL model. In our work, we use BOP to solve the
optimization of the CIL model and the parameterized Class-Incremental Masking
(CIM) model, where CIM is a plugin branch (in the CIL model), using few extra
parameters. The process of BOP is quick and efficient.

##  3 Preliminary

The following is the training pipeline of standard CIL with few-shot
exemplars. Assume there are $N$ learning phases. In the $1$-st phase, we load
data $\mathcal{D}_{1}$ containing all training samples of $c_{1}$ classes, and
use $\mathcal{D}_{1}$ to train the initial classification model
$(\theta_{1},\omega_{1})$, where $\theta_{1}$ and $\omega_{1}$ denote the
parameters of the feature extractor and classifier, respectively. When the
training is done, we evaluate the model performance on the test samples of
$c_{1}$ classes. Before the $2$-nd phase, we discard most of the training
samples due to the strict memory budget of CIL. In other words, we preserve
only a handful of training samples $\mathcal{E}_{1}$ (i.e., exemplars) in the
memory, selected from $\mathcal{D}_{1}$. A common method for selecting
exemplars is called feature herding [37] and has been used in many related
works [28, 48, 45, 44]. We adopt it, too, in this work. In the $i$-th phase
($i\geq 2$), we load all exemplars
${\mathcal{E}}_{1:i-1}=\mathcal{E}_{1}\cup\dots\cup{\mathcal{E}}_{i-1}$ from
the memory and initialize the current model $(\theta_{i},\omega_{i})$ by the
previous model $(\theta_{i-1},\omega_{i-1})$. We use $\mathcal{E}_{1:i-1}$ and
the new coming data $\mathcal{D}_{i}$ (containing $c_{i}$ new classes) to
train $(\theta_{i},\omega_{i})$.Then, we evaluate the current model using a
test set of all $\sum_{j=1}^{i}c_{j}$ classes seen so far. After that, we
discard most of the training samples in $\mathcal{D}_{i}$, and leave few-shot
exemplars ${\mathcal{E}}_{i}$ in the memory. It is clear that this discarding
causes a strong data imbalance between old and new coming classes in the
subsequent phase. In the following, we introduce our solution to this problem.

##  4 Methodology

Figure 2: The proposed compression pipeline. Given an image, we extract its
CAM-based (baseline) or CIM-based (ours) mask, threshold it to be a 0-1 mask
with a fixed threshold $\tau$, and generate a tight bounding box (bbox) that
covers all masked pixels. Then, we downsample the pixels outside the bbox and
sum the downsampled image up to the masked image to generate the final
compressed image.

As illustrated in Figure 1d, we alleviate the data imbalance problem by saving
a larger number of compressed exemplars for old classes, where we leverage
pixel-selective compression, i.e., downsample only non-discriminative pixels.
To achieve this, the ideal case is that we have the pixel-level localization
of foreground objects. However, the realistic case is that such localization
labels are expensive, most CIL benchmarks do not have the labels, and it is
not fair to compare with other CIL methods if using the labels. Without extra
labeling, we introduce a CAM-based mask generation method, and based on it, we
provide a baseline solution to pixel-selective compression in Section 4.1. The
problem of mask generation in CIL is that the optimal generation
hyperparameters such as hard thresholds are changing in the dynamic
environment (with the increasing number of classes and phases). It is thus
desirable to have an adaptive mask generation process. To this end, we propose
class-incremental masking (CIM)—a learnable mask generation model, in Section
4.2.

###  4.1 CAM-based Compression Pipeline

Generating pixel-level labels for large-scale datasets, e.g., ImageNet [10],
is non-trivial. Using class activation maps (CAM) is a naïve solution with
little computation costs. Its key idea is to make use of the activation of the
classification model itself: on the feature maps, activated pixels are more
discriminative than non-activated ones for recognizing the object, where
“activated” means _of high activation values_ and _with strong correlation
with the classification weights of the object_. After localizing activated
pixels by CAM, we can generate a $0$-$1$ mask on them, e.g., by hard
thresholding their normalized values, and then upsample the mask to the size
of the input image [51].

From CAM to $0$-$1$ Mask. We extract CAM in the following steps. Given an
image $x$ from $\mathcal{D}_{i}$ and its ground truth class label $y$, let
$F(x;\theta_{i})$ represent the feature block output by the feature extractor
$\theta_{i}$, and $\omega_{i,y}$ for the classification weights of class $y$
in the classifier $\omega_{i}$. The CAM of $x$ is:

$\mathcal{M}^{\textrm{CAM}}=\frac{A-\min{(A)}}{\max{(A)}-\min{(A)}},\
A={\omega_{i,y}^{\top}F(x;\theta_{i})},$ (1)

where $\min(\cdot)$ and $\max(\cdot)$ operations are used for normalization.
Then, we upsample $\mathcal{M}^{\textrm{CAM}}$ to the size of image $x$ and
use the same notation. Each value in $\mathcal{M}^{\textrm{CAM}}$ denotes the
activation strength of the model at a specific pixel location. Following the
way of generating $0$-$1$ masks in weakly-supervised semantic segmentation
works [12, 8, 23], we apply a hard threshold $\tau$ (between $0$ and $1$) over
all values of $\mathcal{M}^{\textrm{CAM}}$, and get the $0$-$1$ mask
$\mathcal{M}^{\tau}$:
$\mathcal{M}^{\tau}\\!=\\!\mathbb{I}(\mathcal{M}^{\textrm{CAM}}\\!>\\!\tau)$,
where $\mathbb{I}(\cdot)$ is the indicator function. In $\mathcal{M}^{\tau}$,
$1$s indicate the locations of discriminative pixels, e.g., foreground pixels,
based on which the model makes the prediction. While $0$s indicate mostly
background pixels or non-discrimination pixels that can be downsampled as they
make little contribution to the prediction.

After generating $0$-$1$ masks, it is ideal to keep them in the memory as meta
information of compression. However, this is not efficient or feasible in CIL.
There are two reasons. 1) The space for saving image-size masks is non-
negligible. Each mask pixel is a one-bit boolean value, and one mask takes
around $\frac{1}{3\times 8}=\frac{1}{24}$ of the memory of one RGB image. 2)
The mask involves activated regions with irregular shapes. It is thus non-
trivial to perform any standard downsampling algorithm [35] on the remaining
regions.

From $0$-$1$ Masks to Bounding Boxes (BBox). A simple workaround is to
generate a tight bounding box (bbox) to cover the positions of $1$s in
$\mathcal{M}^{\tau}$, and use the bbox for compression. Specifically, given
$\mathcal{M}^{\tau}$, we obtain the coordinate representation of the bounding
box as:

$\mathcal{B}=[\min h,\min w;\max h,\max w]_{(h,w):\mathcal{M}^{\tau}(h,w)=1},$
(2)

where $h$ and $w$ denote the vertical and horizontal coordinates of the $1$ on
$\mathcal{M}^{\tau}$, respectively. We highlight that $\mathcal{B}$ consists
of four integers only and takes negligible memory overhead compared to
$\mathcal{M}^{\tau}$. In addition, we “reshape” the irregular shape (of the
activated region) in $\mathcal{M}^{\tau}$ into a rectangular $\mathcal{B}$, so
our downsampling operation on the pixels outside the rectangular becomes easy.

Compression with BBox. Given the image and its bbox on foreground, the
compression is implemented by downsampling pixels outside the bbox.
Specifically, as illustrated in Figure 2, we compress the image $x$ to
$\tilde{x}$ as follows,

$\tilde{x}=\mathcal{M}^{\mathcal{B}}\odot x+(1-\mathcal{M}^{\mathcal{B}})\odot
x_{\eta},$ (3)

where $\mathcal{M}^{\mathcal{B}}$ is the binary mask according to
$\mathcal{B}$, i.e., the values of $\mathcal{M}^{\mathcal{B}}$ are $1$ inside
$\mathcal{B}$, and $0$ otherwise. $x_{\eta}$ is the fully downsampled version
of $x$ with a downsampling ratio $\eta$ ($\eta>1$), $\odot$ denotes the
element-wise product, and $+$ denotes the element-wise addition. Both $\odot$
and $+$ are applied independently on each RGB channel.

The memory allocated for the compressed image $\tilde{x}$ is as follows,

$\displaystyle m_{\tilde{x}}$
$\displaystyle=\frac{H_{\mathcal{B}}W_{\mathcal{B}}}{HW}+\frac{1}{\eta}\Big{(}1-\frac{H_{\mathcal{B}}W_{\mathcal{B}}}{HW}\Big{)}$
(4)
$\displaystyle=1-\Big{(}1-\frac{1}{\eta}\Big{)}\cdot\Big{(}1-\frac{H_{\mathcal{B}}W_{\mathcal{B}}}{HW}\Big{)},$

where $H_{\mathcal{B}}$ and $W_{\mathcal{B}}$ are the height and width of
$\mathcal{B}$, respectively. $H$ and $W$ are the height and width of the
original image $x$, respectively. $m_{\tilde{x}}$ is always smaller than $1$
where $1$ denotes the memory unit of saving one original image $x$. Therefore,
we can save a larger number of compressed exemplars within the same memory
budget. We denote the set of compressed exemplars in the $i$-th phase as
$\tilde{{\mathcal{E}}}_{i}$.

Compression Artifacts. The above compression introduces artifacts to the
compressed images, i.e., there is a resolution mutation around bounding box
edges. From the perspective of spectrum analysis [21, 5], such mutation
carries noisy and high-frequency components and impairs the model training in
subsequent phases. We mitigate the effect of these artifacts by implementing
the following data augmentation: in each training epoch, we transform a random
subset of $\mathcal{D}_{i}$ into compressed images with CAM-based bounding
boxes using the same downsampling ratio. Using this augmentation enables the
model to “simulate” the training with compressed images and learn to be
invariant to compression artifacts.

###  4.2 Class-Incremental Masking (CIM)

Ideally, the mask generation process needs to adjust at different phases in
CIL environments. The process involves two hyperparameters: the masking
threshold and the choice of network activation functions. First, for the
threshold, searching for its optimum (for all classes) is not trivial. Grid
search is intuitive, but it is computationally expensive when the number of
classes increases in CIL. Second, for the activation function, the standard
network of CIL methods uses ReLU [34, 37] and does not optimize it. We solve
this issue by applying a learnable activation function in addition to the
existing ReLU function in the CIL model. Physically, we have one neural
network, while logically, we have two models to learn (in each incremental
phase): the conventional CIL model with ReLU activations, and the adaptive
mask generation model with learnable activations. We thus call our method
class-incremental masking (CIM) based CIL. In the following, we elaborate on
the network design and optimization pipeline.

Network Design.

Figure 3 demonstrates an example network architecture in our CIM-based CIL.
The proposed CIM extends the network backbone by _logically_ adding a network
branch, where only the activation functions are learnable (e.g., Padé
Activation Units (PAU444PAU uses a rational function of given degrees $m$ and
$n$, i.e., $(a_{0}+a_{1}x+\cdots+a_{m}x^{m})/(1+b_{1}x+\cdots+b_{n}x^{n})$,
and can be parameterized by $a_{0},\cdots,a_{m}$ and $b_{1},\cdots,b_{n}$.,
[33])) and the parameters of weight layers are copied from the original
branch.

This design is motivated by the works of He et al. [16] and Bochkovskiy et al.
[2], which indicate that layers with learnable activation functions can
flexibly process object (localization) information at different network
blocks. The difference is that we apply this flexibility to achieve adaptive
mask generation for different CIL phases.

We denote the CIM parameters (i.e., the parameters in learnable activation
functions) in $i$-th phase as $\phi_{i}$. We optimize the CIL model
$(\theta_{i},\omega_{i})$ and the CIM model $\phi_{i}$ via a global BOP, as
elaborated below.

Figure 3:  Our CIM installs an “extension” on the backbone (e.g., ResNet-18
[17, 37] with four blocks) by adding a learnable activation function (e.g.,
PAU [33]) at the position of the original activation function (i.e., ReLU
[34]). Such “extension” results in a new network branch (in green), whose
weight layer parameters are directly copied from the original branch (in
blue).

Input: New data $\mathcal{D}_{i}$; old compressed exemplars
$\tilde{{\mathcal{E}}}_{0:i-1}$ ($\tilde{{\mathcal{E}}}_{0}=\varnothing$);
last-phase CIL model $(\theta_{i-1},\omega_{i-1})$ and CIM model $\phi_{i-1}$
($\theta_{0}$, $\omega_{0}$ are random parameters, $\phi_{0}$ is set to ReLU).

Output: New compressed exemplars $\tilde{{\mathcal{E}}}_{i}$; updated CIL
model $(\theta_{i},\omega_{i})$ and CIM model $\phi_{i}$.

1 Initialize $(\theta_{i},\omega_{i})$ with $(\theta_{i-1},\omega_{i-1})$;

2 Initialize $\phi_{i}$ with $\phi_{i-1}$;

3 for  _epochs_ do

4 Train $(\theta_{i},\omega_{i})$ using
$\tilde{{\mathcal{E}}}_{0:i-1}\cup\mathcal{D}_{i}$ by Eq. 5;

5 Compress $\mathcal{D}_{i}$ into $\tilde{\mathcal{D}}^{\phi}_{i}$ using
$\phi_{i}$;

6 Temporarily update $\theta_{i}$ to $\theta_{i}^{+}$ using
$\tilde{{\mathcal{E}}}_{0:i-1}\cup\tilde{\mathcal{D}}^{\phi}_{i}$ by Eq. 7;

7 Learn $\phi_{i}$ using $\mathcal{D}_{i}$ by Eq. 9;

8

9 end for

10Compress $\mathcal{D}_{i}$ into $\tilde{\mathcal{D}}_{i}$ using the learned
$\phi_{i}$ by Eq. 3;

Select exemplars $\tilde{\mathcal{E}}_{i}$ from $\tilde{\mathcal{D}}_{i}$ by,
e.g., herding [37].

Algorithm 1 CIM-based CIL ($i$-th phase, $i\\!\geq\\!1$)

Optimization Pipeline. We demonstrate the overall optimization flow in
Algorithm 1, which consists of two levels of optimization: task-level and
mask-level—the former one for CIL and the latter one for CIM. Note that to
maintain an unified notation, we further define
$\tilde{{\mathcal{E}}}_{0}=\varnothing$.

1) Task-level Optimization. This level aims to optimize the CIL model
$(\theta_{i},\omega_{i})$ to address the CIL task at hand. It can be written
as:

$\displaystyle(\theta_{i},\omega_{i})\leftarrow(\theta_{i},\omega_{i})-\lambda\nabla_{(\theta,\omega)}\mathcal{L}_{\textrm{CIL}}(\tilde{\mathcal{E}}_{0:i-1}\\!\cup\\!\mathcal{D}_{i};\theta_{i},\omega_{i}),$
(5)

where $\lambda$ is the learning rate. We follow the implementation of CIL
training loss $\mathcal{L}_{\textrm{CIL}}$ in baseline methods [24, 52, 48,
44]. This means that we use different training losses when plugging CIM into
different baseline methods.

2) Mask-level Optimization. This level aims to optimize the CIM model
$\phi_{i}$ to produce adaptive compression masks. It is formulated as a local
BOP:

$\displaystyle\min_{\phi_{i}}\left[{\mathcal{L}_{\textrm{val}}(\mathcal{D}_{i};\theta^{*}_{i},\omega_{i})+\mu\mathcal{R}(\phi_{i})}\right],$
(6a)
$\displaystyle~{}\mathrm{s.t.}~{}\theta_{i}^{*}=\operatorname*{arg\,min}_{\theta_{i}}{\mathcal{L}_{\textrm{train}}(\tilde{\mathcal{E}}_{0:i-1}\cup\tilde{\mathcal{D}}_{i}(\phi_{i});\theta_{i},\omega_{i})}.$
(6b)

Eq. 6b denotes an inner-level optimization. It trains $\theta_{i}$ with the
data $\tilde{\mathcal{D}}_{i}(\phi_{i})$ compressed by using $\phi_{i}$, and
converges as $\theta_{i}^{*}$. Eq. 6a denotes an outer-level optimization. It
is based on the validation loss derived by $\theta_{i}^{*}$ on the original
data $\mathcal{D}_{i}$. $\mathcal{R}(\phi_{i})$ is a constraint representing
the memory limitation and $\mu$ is its weight. In the following, we elaborate
on the implementation details for the two levels.

In the inner-level optimization, we train a temporary CIL model with
compressed data. Specifically, we first compress new-class data
$\mathcal{D}_{i}$ into $\tilde{\mathcal{D}}_{i}(\phi_{i})$ using the generated
masks by the CIM model $\phi_{i}$. Then, we implement the inner-level
optimization as a one-step gradient descent (using the CIL training loss) as:

$\theta^{+}_{i}\leftarrow\theta_{i}-\beta_{1}\nabla_{\theta}\mathcal{L}_{\textrm{CIL}}(\tilde{\mathcal{E}}_{0:i-1}\cup\tilde{\mathcal{D}}_{i}(\phi_{i});\theta_{i},\omega_{i}),$
(7)

where $\beta_{1}$ is the learning rate for $\theta_{i}$.

The aim of the outer-level optimization is to optimize $\phi_{i}$ such that
the temporary model $(\theta^{+}_{i},\omega_{i})$ (trained with compressed
data) has a low validation loss on the original data $\mathcal{D}_{i}$. To
achieve this, we back-propagate the loss on the original data
$\mathcal{D}_{i}$ to update $\phi_{i}$ as:

$\phi_{i}\leftarrow\phi_{i}-\beta_{2}\nabla_{\phi}\left[\mathcal{L}_{\textrm{CE}}(\mathcal{D}_{i};\theta^{+}_{i},\omega_{i})+\mu\mathcal{R}(\phi_{i})\right],$
(8)

where $\mathcal{L}_{\textrm{CE}}$ denotes softmax cross-entropy loss and
$\beta_{2}$ is the learning rate for $\phi_{i}$. This trains $\phi_{i}$ to
capture the most discriminative features of new-class images. The constraint
$\mathcal{R}(\phi_{i})$ is implemented as a $\ell_{2}$-regularization term on
the generated mask by $\phi_{i}$. The motivation for the regularization term
is to make the mask coverage smaller, thus the compressed images take less
memory.

We empirically observe that by the above optimization flow, the output
activation maps by $\phi_{i}$ are easy to collapse, i.e., different images
have the same map. To solve this issue, we add a cross-entropy loss term about
$\phi_{i}$ in Eq. 8 to regularize it to produce image-specific activation
maps:

$\displaystyle\phi_{i}\leftarrow\phi_{i}-\beta_{2}\nabla_{\phi}[\mathcal{L}_{\textrm{CE}}($
$\displaystyle\mathcal{D}_{i};\theta^{+}_{i},\omega_{i})+\mu\mathcal{R}(\phi_{i})$
(9) $\displaystyle+\mu^{\prime}$
$\displaystyle\mathcal{L}_{\textrm{CE}}(\tilde{\mathcal{E}}_{0:i-1}\cup\mathcal{D}_{i};\theta_{i},\phi_{i},\omega_{i})],$

where $\mu^{\prime}$ is the weight.

Limitations. Our CIM learns to generate adaptive masks for exemplar
compression in CIL. It has three limitations that are left as future work. 1)
It is not able to adjust any previous-phase exemplars, as the validation data
(the original data of these exemplars) are not accessible anymore. 2) It
introduces hundreds of activation parameters to the CIL model, although this
is not a significant overhead compared to model parameters. Please check
detailed overhead analyses in the supplementary materials. 3) Image
compression is not that meaningful for low-resolution datasets (e.g.,
$32\\!\times\\!32$ CIFAR-100) It is because the memory taken by the
compression parameters (e.g., the parameters of CIM) and the RGB pixels of a
low-resolution image are comparable. Using the memory to save more images is
more meaningful.

##  5 Experiments

We incorporate CIM into two baseline CIL methods (i.e., DER [48] and FOSTER
[44]) and boost their model performances consistently on three datasets.
Below, we introduce datasets and experiment settings (Section 5.1), followed
by results and analyses (Section 5.2)

###  5.1 Experimental Settings

Datasets. We conduct experiments on three standard CIL benchmarks with high-
resolution images. 1) Food-101 [3] consists of $101$ food categories with
$750$ training and $250$ test samples per category. All images have a maximum
side length of $512$ pixels. 2) ImageNet-1000 [10] is a large-scale dataset
with $1,\\!000$ classes and each class has around $1,\\!300$ training and $50$
test samples. 3) ImageNet-100 is a 100-class subset randomly sampled from
ImageNet-1000 with a fixed NumPy [15] random seed ($1993$), following [18]. We
provide other details of these datasets, e.g., image sizes and pre-processing
methods, in the supplementary materials.

Protocols. We use two protocols: learning from scratch (LFS) and learning from
half (LFH), following recent CIL works [48, 44]. In LFS, the model observes
the same number of classes in all $N$ phases, where $N$ is optionally $5$,
$10$, and $20$. In LFH, the model is trained on half of the classes (e.g.,
$500$ classes for ImageNet-1000) in the $1$-th phase. Then, it learns the
remaining classes evenly in the subsequent $N$ phases, where $N$ can be $5$,
$10$, and $25$. In both protocols, after the training of each phase we
evaluate the resultant model on the test data of all seen classes. Our final
report includes the average accuracy over all phases and the last-phase
accuracy which indicates the degree of model forgetting. We run each
experiment three times and report the average results.

Memory Budget. There are two memory budget555Please note that we measure the
memory by the number of original images. Each compressed exemplar in CIM takes
less memory than the original image (Eq. 4), resulting in more exemplars in
the same memory. settings. 1) In the “fixed” setting, we remove some old-class
exemplars when new exemplars from the current phase are added in the memory to
maintain the “fixed memory budget”. In this setting, we set the total memory
to be $2,\\!020$ samples for Food-101 and $2,\\!000$ samples for ImageNet-100.
For ImageNet-1000, we have two options—$5,\\!000$ samples and $20,\\!000$
samples. 2) In the “growing” setting, a constant memory budget is allocated
for each class across all phases and hence extra memory is appended when new
classes come. In this setting, we set the budget to be $20$ samples per class
for all datasets. Following [48, 44], we apply the “fixed” setting in LFS
experiments and the “growing” setting in LFH experiments.

Method | Learning from Scratch (LFS) |  | Learning from Half (LFH)
---|---|---|---
Food-101 |  | ImageNet-100 |  | Food-101 |  | ImageNet-100
$N$=5 | 10 | 20 |  | 5 | 10 | 20 |  | 5 | 10 | 25 |  | 5 | 10 | 25
iCaRL [37] | 69.66 | 62.18 | 56.70 |  | 73.90 | 67.06 | 62.36 |  | 60.13 | 53.42 | 46.87 |  | 62.53 | 59.88 | 52.97
WA [50] | 70.94 | 63.69 | 58.45 |  | 74.64 | 68.62 | 63.20 |  | 63.55 | 57.60 | 52.48 |  | 65.75 | 63.71 | 58.34
PODNet [13] | 68.03 | 61.24 | 47.38 |  | 72.14 | 63.96 | 53.69 |  | 75.37 | 70.01 | 65.32 |  | 75.54 | 74.33 | 68.33
AANets [26] | 69.46 | 61.59 | 48.83 |  | 72.98 | 65.77 | 55.36 |  | 76.07 | 71.22 | 66.93 |  | 76.96 | 75.58 | 71.78
1-16[4pt/2pt] DER [48] | 73.88 | 70.76 | 64.39 |  | 78.50 | 76.12 | 73.79 |  | 78.13 | 73.45 | - |  | 79.08 | 77.73 | -
DER w/ ours | 75.63 | 73.09 | 69.17 |  | 79.63 | 77.57 | 75.36 |  | 79.25 | 75.76 | - |  | 80.30 | 79.05 | -
1-16[4pt/2pt] FOSTER [44] | 75.03 | 72.72 | 66.73 |  | 79.93† | 76.55† | 74.49 |  | 79.08 | 75.07 | 68.08 |  | 80.07† | 77.54 | 72.40*
FOSTER w/ ours | 76.44 | 74.85 | 70.20 |  | 80.58 | 77.94 | 75.23 |  | 79.76 | 76.86 | 70.50 |  | 80.93 | 78.66 | 75.74

  * †

The paper of FOSTER [44] did not report the numerical results for $N$=$5$/$10$
(LFS) and $N$=$5$ (LFH) on ImageNet-100. We run these experiments using the
public code (released by authors) and report the reproduced results.

  * *

Our reproduced result ($72.40$) is significantly higher than the original
result ($69.34$) reported in the paper of FOSTER.

Table 1: Average accuracies (%) of two top-performing CIL methods [48, 44]
with and without our CIM-CIL plugged-in, and other four baselines [37, 50, 13,
26], on two datasets (Food-101 and ImageNet-100) and using two protocols
(learning from scratch (LFS) and learning from half (LFH)). Due to the space
limits, we report the $95\%$ confidence intervals for these results in the
supplementary materials.

Implementation Details. Our implementation is based on the standard deep
learning library PyTorch [36] and image processing library OpenCV [4].
Following [48, 45, 44], we use an $18$-layer ResNet [17] as the network
backbone $\theta$ and a fully-connected layer as the classifier $\omega$ in
all experiments. We use the same CIL training hyperparameters as in related
works [18, 48, 44] for fair comparison: 1) there are $200$ epochs in $1$-st
phase and $170$ epochs in the subsequent phases; 2) the learning rate
$\lambda$ is initialized as $0.1$ and decreases to zero with a cosine
annealing scheduler [29]; 3) the SGD optimizer is deployed, with momentum
factor set to $0.9$ and weight decay set to $0.0005$. For compression-related
hyperparameters, we set the masking threshold $\tau$ as $0.6$ and the
downsampling ratio $\eta$ as $4.0$. To build the CIM model, we apply PAUs with
degrees $m=5$ and $n=4$ as learnable activation layers. For the optimization
of the CIM model $\phi$ (i.e., the mask-level optimization), we initially set
$\beta_{1}$ as $0.1$ and $\beta_{2}$ as $0.01$ and reduce them to zero
following the scheduler of $\lambda$. $\mu$ and $\mu^{\prime}$ is set to $0.1$
and $0.2$, respectively. To smooth the training, we clip the gradient norm of
$\phi$ to be no more than $1$. _We report the result of hyperparameter
sensitivity analysis in the supplementary materials._

###  5.2 Results and Analyses

Memory Budget | Method |  $N$=5 |  |  $N$=10
---|---|---|---|---
Avg. | Last |  | Avg. | Last
$M\\!=\\!20k$ | iCaRL [37] | 44.36 | 27.78 |  | 38.40 | 22.70
WA [50] | 58.37 | 50.62 |  | 54.10 | 45.66
DER [48] | 67.49 | 59.75 |  | 66.73 | 58.62
2-9[4pt/2pt] | FOSTER [44] | 69.21 | 64.88 |  | 68.34 | 60.14
| FOSTER w/ ours | 69.93 | 66.05 |  | 69.53 | 62.07
1-9[4pt/2pt] $M\\!=\\!5k$ | FOSTER | 57.19 | 49.42 |  | 54.72 | 44.96
FOSTER w/ ours | 61.37 | 54.46 |  | 59.48 | 50.83
Table 2: Average and last accuracies (%) on ImageNet-1000 of FOSTER with and
without our method plugged-in, and other three baselines [37, 50, 48]. We show
two memory budgets, $M\\!=\\!20k$ (upper block) and $M\\!=\\!5k$ (lower
block), in the LFS setting.

Comparing with the State-of-the-art. In Table 1, we summarize the experimental
results on two datasets (Food-101 and ImageNet-100) and in two CIL protocols
(LFS and LFH). From the table, we have the following observations. 1) Our CIM-
based CIL consistently improves the state-of-the-art method FOSTER [44] with
clear margins in all settings. E.g., our method surpasses it by an average of
$1.4$ percentage points on ImageNet-100, and $2.0$ percentage points on
Food-101. 2) Our CIM-based CIL achieves more significant improvements when $N$
becomes larger, e.g., on ImageNet-100 (LFH), our method improves FOSTER by
$0.9$ and $3.3$ percentage points when $N$=$5$ and $N$=$25$, respectively. 3)
Our CIM-based CIL achieves greater improvements consistently on Food-101 (than
ImageNet-100). It improves baselines by $2.1$ percentage points on Food-101,
while the improvement is $1.4$ on ImageNet-100 ($N$=$10$, LFS). It shows that
our method is particularly effective when the representative visual cues of a
class are from some of its components, e.g., the “cream” of the class “cake”.

Table 2 shows the results on the large-scale dataset ImageNet-1000 in
different memory settings ($M=20k$ and $M=5k$). We can see that our CIM-based
CIL improves FOSTER consistently. It is impressive that it achieves more
improvements in the more strict memory setting ($M=5k$). Specifically, it
boosts the average accuracy of FOSTER by $4.5$ percentage points when $M=5k$,
significantly higher than that of $M=20k$ ($1.0$).

Ablation Method | Food-101 |  | ImageNet-100
---|---|---|---
$N$=10 | 20 |  | 10 | 20
1 Baseline | 72.72 | 66.73 |  | 76.55 | 72.37
2 Artifact Aug. | 71.38 | 66.03 |  | 75.63 | 71.45
1-6[4pt/2pt] 3 Full Comp. | 73.03 | 67.38 |  | 76.92 | 73.26
4 Random Acti. | 73.10 | 67.54 |  | 76.88 | 73.54
5 Center Acti. | 73.29 | 67.88 |  | 76.78 | 73.82
6 Class Acti. | 73.76 | 68.65 |  | 77.21 | 74.67
1-6[4pt/2pt] 7 Phase-wise $\tau$ | 73.83 | 69.17 |  | 77.06 | 74.78
8 Joint Train | 73.44 | 69.01 |  | 77.34 | 74.59
9 BOP (ours) | 74.85 | 70.20 |  | 77.94 | 75.23
1-6[4pt/2pt] 10 LastBlock Only | 74.55 | 69.87 |  | 77.72 | 74.86
11 Fg Compressed | 75.02 | 70.13 |  | 77.87 | 75.46
Table 3: Average accuracies (%) of different ablation methods. The experiments
are conducted in the LFS setting.

Ablation Study. Table 3 shows the ablation results. _First block: baselines._
Row 1 is for the baseline FOSTER [44]. Row 2 shows the results of adding
artifact augmentation (see Section 4.1). It shows directly apply this
augmentation does not improve and even impair the model. _Please note the
models in below blocks all use this augmentation_. _Second block: activation
methods._ Rows 3-6 show the results of using different activation methods to
compress exemplars. Row 3 is to downsample all pixels (i.e., no region is
activated). Row 4 is to randomly select activation regions. Row 5 is to
activate only the center region ($\frac{1}{4}$ of the original image), while
row 6 is to use naive CAM. Comparing them to row 1, we can see that using
naively compressed exemplars can improve CIL models. Row 4 outperforms rows
3-5, validating that it is more reliable to use the model’s activation to
generate compressed exemplars. _Third block: optimization methods._ Rows 7-9
are on top of Row 6 and are the results of applying different optimization
strategies. Row 7 is to manually select $\tau$ using a held-out set ($10\%$ of
the dataset). Row 8 is to jointly train CIL and CIM models (for each input
batch). Row 9 is the proposed method of using a global BOP. _Fourth block: two
variants of CIM-based CIL._ Rows 10-11 are two variants of row 9. In Row 10,
only the activation layers in the last block of CIM are learnable, and
previous blocks use ReLU. Compared to row 9, row 10 shows slightly worse
performance. Row 11 shows the version of adding a weak downsampling
($\eta^{\prime}=2.0$) on discriminative regions, based on which more
compressed exemplars are saved. It results in comparable performance to row 9
but increases costs.

Method | ImageNet-100 |  | ImageNet-1000
---|---|---|---
$N$=6 | 11 | 26 |  | 6 | 11 | 26
LUCIR _baseline_ | 71.22 | 69.67 | 67.45 |  | 65.23 | 62.43 | 59.88
w/ Mnemonics | 73.30 | 72.17 | 71.50 |  | 66.15 | 63.12 | 63.08
w/ MRDC | 73.62 | 72.81 | 70.44 |  | 67.67 | 65.60 | 62.74
w/ ours | 74.05 | 73.76 | 72.84 |  | 68.03 | 66.54 | 63.77
Table 4: Comparing with Mnemonics [28] and MRDC [45]. We plug each of them in
baseline LUCIR [18] for fair comparison.

Comparing with Other Compression-based Methods. Table 4 shows our results
comparing to two compression-based methods: Mnemonics [28] and MRDC [45]. We
can see that our method consistently outperforms them in all settings. This is
because our method does not sacrifice the discriminativeness of exemplars
while improving the number (variance) of exemplars in a phase-adaptive manner.
However, the two related methods either keep a fixed number of exemplars in
the memory [28] or use uniform image compression without considering the
properties of specific classes in different incremental phases [45].

Visualizations (CAM vs. CIM). Figure 4 gives two visualization examples,
“Afghan hound” and “indigo bird”, each with the activation map as well as the
bounding boxes. The first column shows their respective confusing classes
appearing in earlier phases. CIM learns to focus on the discriminative (i.e.,
dissimilar to confusing classes) regions.

Figure 4: Visualizations (CAM vs. CIM). The experimental setting is $N$=$10$ (LFS) on ImageNet-100.  Metric | Small | Middle | Large
---|---|---|---
Mean of #Exemplars | 39.40 | 38.30 | 34.77
1-4[4pt/2pt] Last Acc. (%, baseline) | 66.13 | 68.40 | 69.93
Last Acc. (%, ours) | 70.00 | 71.10 | 72.26
Improvement (%) | +3.87 | +3.65 | +2.33
Table 5: Results for small, middle and large objects in the setting of
$N$=$10$ (LFS) on ImageNet-100. “Mean of #Exemplars” denotes the average
number of saved exemplars by our method. The baseline (FOSTER [44]) has this
number as $20$ for all classes.

Results of Different-Size Objects. Table 5 shows the results for small,
middle, and large objects. These size categorization is according to ImageNet
Object Localization Challenge [20]. We calculate the bbox coverage for each
class and take top $30$ classes with highest coverages as “large”, rear $30$
classes with lowest coverages as “small” and the rest $40$ classes as
“middle”. It is intriguing that our method achieves the highest improvement
(over baseline) for small objects. Our explanation is that small objects
benefit more from image compression (than large ones), as their images contain
more background pixels to downsample.

##  6 Conclusions

We introduced a novel exemplar compression method for CIL, allowing us to save
more representative exemplars but not increase memory budget. We achieved this
compression by downsampling non-discriminative pixels of the image bounded by
CAM masks. To generate adaptive masks, we proposed a novel method CIM that
explicitly parameterizes a mask generation model and optimizes it in an end-
to-end manner across incremental phases. Our method achieves consistent
performance improvements over multiple baselines and can be taken as a
flexible plug-and-play module.

Acknowledgements. This research was supported by the A*STAR under its AME YIRG
Grant (Project No. A20E6c0101). The author gratefully acknowledges the support
by the Lee Kong Chian (LKC) Fellowship fund awarded by Singapore Management
University.

## References

  * [1] Davide Abati, Jakub Tomczak, Tijmen Blankevoort, Simone Calderara, Rita Cucchiara, and Babak Ehteshami Bejnordi.  Conditional channel gated networks for task-aware continual learning.  In CVPR, 2020.
  * [2] Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao.  Yolov4: Optimal speed and accuracy of object detection.  arXiv, 2020.
  * [3] Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool.  Food-101 – mining discriminative components with random forests.  In ECCV, 2014.
  * [4] Gary Bradski.  The opencv library.  Dr. Dobb’s Journal: Software Tools for the Professional Programmer, 2000.
  * [5] Kenneth R Castleman.  Digital image processing.  Prentice Hall Press, 1996.
  * [6] George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A Efros, and Jun-Yan Zhu.  Dataset distillation by matching training trajectories.  In CVPR, 2022.
  * [7] Can Chen, Xi Chen, Chen Ma, Zixuan Liu, and Xue Liu.  Gradient-based bi-level optimization for deep learning: A survey.  arXiv, 2022.
  * [8] Zhaozheng Chen, Tan Wang, Xiongwei Wu, Xian-Sheng Hua, Hanwang Zhang, and Qianru Sun.  Class re-activation maps for weakly-supervised semantic segmentation.  In CVPR, 2022.
  * [9] Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le.  Autoaugment: Learning augmentation strategies from data.  In CVPR, 2019.
  * [10] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In CVPR, 2009.
  * [11] Prithviraj Dhar, Rajat Vikram Singh, Kuan-Chuan Peng, Ziyan Wu, and Rama Chellappa.  Learning without memorizing.  In CVPR, 2019.
  * [12] Zhang Dong, Zhang Hanwang, Tang Jinhui, Hua Xiansheng, and Sun Qianru.  Causal intervention for weakly supervised semantic segmentation.  In NeurIPS, 2020.
  * [13] Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle.  Podnet: Pooled outputs distillation for small-tasks incremental learning.  In ECCV, 2020.
  * [14] Chelsea Finn, Pieter Abbeel, and Sergey Levine.  Model-agnostic meta-learning for fast adaptation of deep networks.  In ICML, 2017.
  * [15] Charles R Harris, K Jarrod Millman, Stéfan J Van Der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J Smith, et al.  Array programming with numpy.  Nature, 2020.
  * [16] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Delving deep into rectifiers: Surpassing human-level performance on imagenet classification.  In ICCV, 2015.
  * [17] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In CVPR, 2016.
  * [18] Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin.  Learning a unified classifier incrementally via rebalancing.  In CVPR, 2019.
  * [19] Shenyang Huang, Vincent François-Lavet, and Guillaume Rabusseau.  Neural architecture search for class-incremental learning.  arXiv, 2019.
  * [20] Imagenet object localization challenge.  https://www.kaggle.com/competitions/imagenet-object-localization-challenge/.
  * [21] Anil K Jain.  Fundamentals of digital image processing.  Prentice-Hall, Inc., 1989.
  * [22] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al.  Overcoming catastrophic forgetting in neural networks.  PNAS, 2017.
  * [23] Jungbeom Lee, Eunji Kim, and Sungroh Yoon.  Anti-adversarially manipulated attributions for weakly and semi-supervised semantic segmentation.  In CVPR, 2021.
  * [24] Zhizhong Li and Derek Hoiem.  Learning without forgetting.  PAMI, 2017.
  * [25] Yaoyao Liu, Yingying Li, Bernt Schiele, and Qianru Sun.  Online hyperparameter optimization for class-incremental learning.  In AAAI, 2023.
  * [26] Yaoyao Liu, Bernt Schiele, and Qianru Sun.  Adaptive aggregation networks for class-incremental learning.  In CVPR, 2021.
  * [27] Yaoyao Liu, Bernt Schiele, and Qianru Sun.  Rmm: Reinforced memory management for class-incremental learning.  In NeurIPS, 2021.
  * [28] Yaoyao Liu, Yuting Su, An-An Liu, Bernt Schiele, and Qianru Sun.  Mnemonics training: Multi-class incremental learning without forgetting.  In CVPR, 2020.
  * [29] Ilya Loshchilov and Frank Hutter.  Sgdr: Stochastic gradient descent with warm restarts.  arXiv, 2016.
  * [30] Dougal Maclaurin, David Duvenaud, and Ryan Adams.  Gradient-based hyperparameter optimization through reversible learning.  In ICML, 2015.
  * [31] Michael McCloskey and Neal J Cohen.  Catastrophic interference in connectionist networks: The sequential learning problem.  In Psychology of Learning and Motivation. Elsevier, 1989.
  * [32] Ken McRae and Phil A Hetherington.  Catastrophic interference is eliminated in pretrained networks.  In Proceedings of the 15h Annual Conference of the Cognitive Science Society, 1993.
  * [33] Alejandro Molina, Patrick Schramowski, and Kristian Kersting.  Padé activation units: End-to-end learning of flexible activation functions in deep networks.  In ICLR, 2019.
  * [34] Vinod Nair and Geoffrey E Hinton.  Rectified linear units improve restricted boltzmann machines.  In ICML, 2010.
  * [35] J Anthony Parker, Robert V Kenyon, and Donald E Troxel.  Comparison of interpolating methods for image resampling.  IEEE Transactions on Medical Imaging, 1983.
  * [36] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.  Pytorch: An imperative style, high-performance deep learning library.  In NeurIPS, 2019.
  * [37] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert.  iCaRL: Incremental classifier and representation learning.  In CVPR, 2017.
  * [38] Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell.  Progressive neural networks.  arXiv, 2016.
  * [39] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In ICCV, 2017.
  * [40] Christian Simon, Piotr Koniusz, and Mehrtash Harandi.  On learning the geodesic path for incremental learning.  In CVPR, 2021.
  * [41] Ankur Sinha, Pekka Malo, and Kalyanmoy Deb.  A review on bilevel optimization: from classical to evolutionary approaches and applications.  IEEE Transactions on Evolutionary Computation, 2017.
  * [42] Qianru Sun, Yaoyao Liu, Tat-Seng Chua, and Bernt Schiele.  Meta-transfer learning for few-shot learning.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 403–412, 2019.
  * [43] Gregory K Wallace.  The jpeg still picture compression standard.  Communications of the ACM, 1991.
  * [44] Fu-Yun Wang, Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan.  Foster: Feature boosting and compression for class-incremental learning.  arXiv, 2022.
  * [45] Liyuan Wang, Xingxing Zhang, Kuo Yang, Longhui Yu, Chongxuan Li, Lanqing Hong, Shifeng Zhang, Zhenguo Li, Yi Zhong, and Jun Zhu.  Memory replay with data compression for continual learning.  In ICLR, 2022.
  * [46] Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu.  Large scale incremental learning.  In CVPR, 2019.
  * [47] Ju Xu and Zhanxing Zhu.  Reinforced continual learning.  In NeurIPS, 2018.
  * [48] Shipeng Yan, Jiangwei Xie, and Xuming He.  Der: Dynamically expandable representation for class incremental learning.  In CVPR, 2021.
  * [49] Friedemann Zenke, Ben Poole, and Surya Ganguli.  Continual learning through synaptic intelligence.  In ICML, 2017.
  * [50] Bowen Zhao, Xi Xiao, Guojun Gan, Bin Zhang, and Shu-Tao Xia.  Maintaining discrimination and fairness in class incremental learning.  In CVPR, 2020.
  * [51] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba.  Learning deep features for discriminative localization.  In CVPR, 2016.
  * [52] Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan.  Co-transport for class-incremental learning.  In Proceedings of the 29th ACM International Conference on Multimedia, 2021.

Supplementary materials

These supplementary materials include implementation details (§1), dataset
details (§2), SOTA $95\%$ CI results (§3), learned PAU results (§4),
compression footprint results (§5), sensitivity analysis results (§6),
overhead analysis results (§7), and hardware information (§8).

##  1 Implementation Details

Downsampling Method. Supplementary to Section 4.1 “Compression with BBox”. For
generating compressed exemplars, we adopted a simple downsampling method (to
apply on non-discriminative pixels) called Nearest Neighbor Interpolation
[35]. Specifically, it replaces the pixel value with that of the nearest
pixel. This can be achieved by calling the cv2.resize() function with the
INTER_NEAREST flag in the standard image processing library OpenCV [4].

Mitigating the Effect of Compression Artifacts. Supplementary to Section 4.1
“Compression Artifacts”. We mitigated the effect of compression artifacts by
applying a augmentation method to new-class data $\mathcal{D}_{i}$ in each
learning phase. Specifically, there were three steps. 1) For all images in
$\mathcal{D}_{i}$, we generated the CAM-based bounding boxes at the beginning
of training and updated them once per $40$ epochs. 2) In each epoch, we
randomly selected a subset of $\mathcal{D}_{i}$. The proportion of the subset
was adjusted according to the training progress—$0$ at the beginning and
increased by $0.1$ every $40$ epochs. 3) Before feeding each image in the
subset (into the CIL model), we downsampled (with ratio $\eta$) the pixels
outside the bounding box (obtained in step 1).

Compressing $\mathcal{D}_{i}$ into $\tilde{\mathcal{D}}_{i}(\phi_{i})$.
Supplementary to Section 4.2 “2) Mask-level Optimization”. Different from the
compressed pipeline introduced in Section 4.1, compression in the inner-level
optimization should involve only differentiable operations to enable gradient
descent. To achieve this, we skipped the steps of adding the threshold and
bounding box and obtained the compressed images by applying continuously
valued masks as follows,

$\displaystyle A(\phi_{i})={\omega_{i,y}^{\top}F(x;\theta_{i},\phi_{i})},$
(S1a)
$\displaystyle\mathcal{M}^{\textrm{CAM}}(\phi_{i})=\frac{A(\phi_{i})-\min{(A(\phi_{i}))}}{\max{(A(\phi_{i}))}-\min{(A(\phi_{i}))}},$
(S1b)
$\displaystyle\tilde{x}(\phi_{i})=\mathcal{M}^{\textrm{CAM}}(\phi_{i})\odot
x+(1-\mathcal{M}^{\textrm{CAM}}(\phi_{i}))\odot x_{\eta}.$ (S1c)

Comparing with Other Compression-based Methods. Supplementary to Table 4. The
code of plugging MRDC [45] into PODNet [13] (which shows the best results in
its original paper) was not released by authors. So we rerun the results of
MRDC when plugging it into LUCIR [18]. The experiments are conducted on the
LFH setting. For a fair comparison, we apply weight transfer operations [42]
in all these experiments following Mnemonics [28].

##  2 Dataset Details

Supplementary to Section 4.1 “Datasets”. We show the details about three
datasets in Table S1. We elaborate the image preprocessing methods applied on
the three datasets in Table S2. Please note that for image preprocessing, we
strictly followed [37, 18, 13, 28, 26, 48, 44] for a fair comparison.

Dataset | #Classes | #Training images | #Test images |  Avg. size
---|---|---|---|---
Food-101 [3] | 101 | 75,750 | 25,250 | 475$\times$496
ImageNet-100 [37] | 100 | 129,395 | 5,000 | 407$\times$472
ImageNet-1000 [10] | 1,000 | 1,281,167 | 50,000 | 406$\times$474
Table S1: Details of the three datasets. The “Avg. size” colume shows the average height$\times$width of images of each dataset. Training transformation | Test transformation | CAM transformation
---|---|---
RandomResizedCrop(224), | Resize(256), | Resize((224,224)),
RandomHorizontalFlip(0.5), | CenterCrop(224), | ToTensor(),
ColorJitter(63/255), | ToTensor(), | Normalize().
ToTensor(), | Normalize(). |
Normalize(). |  |
Table S2: Uniform image preprocessing methods for the three datasets. “CAM
transformation” denotes the preprocessing method for generating CAM [51]. The
mean and standard deviation parameters of Normalize() are omitted. Note that
FOSTER [44] additionally applies AutoAugment [9] in the training
transformation, and we followed it for fair comparison.

##  3 More Results Comparing with the SOTA

Method | Learning from Scratch (LFS) |  | Learning from Half (LFH)
---|---|---|---
Food-101 |  | ImageNet-100 |  | Food-101 |  | ImageNet-100
$N$=5 | 10 | 20 |  | 5 | 10 | 20 |  | 5 | 10 | 25 |  | 5 | 10 | 25
iCaRL [37] | 0.58 | 1.11 | 1.50 |  | 0.98 | 0.64 | - |  | 1.24 | 0.96 | 0.42 |  | 0.86 | 1.48 | 1.37
WA [50] | 0.43 | 0.36 | 2.04 |  | 0.65 | 0.75 | - |  | 0.38 | 0.30 | 1.54 |  | 1.33 | - | -
PODNet [13] | 0.92 | 1.91 | 1.83 |  | 1.08 | 1.87 | - |  | 0.80 | 1.37 | 1.46 |  | 0.29 | 1.05 | 2.77
AANets [26] | 0.83 | 1.56 | 0.74 |  | 1.31 | 1.07 | 2.01 |  | 1.12 | 0.48 | 1.22 |  | 0.53 | 0.74 | 0.81
1-16[4pt/2pt] DER [48] | 0.47 | 0.62 | 0.94 |  | 0.46 | 0.39 | - |  | 0.56 | 0.68 | - |  | 0.51 | - | -
DER w/ ours | 0.28 | 0.75 | 1.10 |  | 0.32 | 0.24 | 0.65 |  | 0.17 | 0.52 | - |  | 0.41 | 0.53 | -
1-16[4pt/2pt] FOSTER [44] | 0.34 | 0.43 | 0.89 |  | 0.51 | 0.53 | - |  | 0.34 | 0.32 | 0.60 |  | 0.07 | - | 0.38
FOSTER w/ ours | 0.45 | 0.22 | 1.25 |  | 0.36 | 0.42 | 0.54 |  | 0.26 | 0.17 | 0.52 |  | 0.44 | 0.18 | 0.61
Table S3: The $95\%$ confidence intervals (%) for the results in Table 1.

Supplementary to Table 1. In Table S3, we report the $95\%$ confidence
intervals corresponding to the numbers in the Table 1 of the main paper.

##  4 Results of Learned PAUs

Figure S1:  The average activation distances between ReLU [34] and the learned
PAUs [33] before and in the last block over all phases. The experimental
setting is $N$=$10$ (LFS) on ImageNet-100. The curves are smoothed with
Gaussian ($\sigma=2$).

Supplementary to Section 5.2 “Results and Analyses”.. Figure S1 shows the
activation distances in different network blocks and phases. We measure the
distance by $\int_{-3}^{3}|f_{\text{PAU}}(x)-f_{\text{ReLU}}(x)|dx$ and use
$601$ interpolated points to approximate the integration value. The learned
PAUs in the last block have larger distances than those in shallow blocks. The
last block mainly encodes high-level semantic information (e.g., “body” of
“dog”), this suggests that the learned PAUs are adjusted to focus on the most
discriminative semantics. Shallow blocks learn to capture low-level features
that are shareable between classification and mask generation. Therefore, the
learned PAUs in these shallow blocks have little adjustment. This motivates
the variant in Row 10 of Table 3 in the main paper.

##  5 Results of Compression Footprints.

Figure S2:  The compression rates resulted from CAM and CIM. The experimental
setting is $N$=$5$ (LFS) on ImageNet-100.

Supplementary to Section 5.2 “Results and Analyses”.. Figure S2 provides the
compression footprints along incremental phases. Comparing with CAM, CIM
learns to produce more conservative compression footprints in later phases.
Our explanation is that more visual cues are required to classify more
classes.

##  6 Results of Sensitivity Analyses

(a) Masking threshold $\tau$.

(b) Downsampling ratio $\eta$.

(c) Inner-level learning rate $\beta_{1}$.

(d) Outer-level learning rate $\beta_{2}$.

(e) $\ell_{2}$ regularization weight $\mu$.

(f) CE regularization weight $\mu^{\prime}$.

Figure S3:  Results of hyperparameter sensitivity.

Supplementary to Section 5.1 “Implementation Details”. In Figure S3, we show
the sensitivity analyses for CAM threshold $\tau$, downsampling ratio $\eta$,
mask-level learning rates $\beta_{1},\beta_{2}$ and two regularization weights
$\mu,\mu^{\prime}$ on ImageNet-100 [37]. For $\tau$ and $\eta$, we also show
the computation overheads.

##  7 Space and Computation Overheads

Supplementary to Section 4.2 “Limitations”. We elaborate on the space overhead
by taking ResNet-18 as an example. We add $17$ PAUs to it, each with $10$
optimizable parameters. So we use $170$ extra parameters in total. This is
negligible compared to $11$ million of network parameters. Besides, we save
bbox along with exemplars in the memory. Each bbox takes around $0.01\%$
memory of a $224\times 224$ RGB image. For computation overhead, our method
needs around $60\%$ extra computations over baseline CIL training, caused by
two factors: 1) BOP between CIL and CIM models; and 2) training on a large
number of compressed exemplars.

##  8 Hardware Information

– CPU: AMD EPYC 7F72 24-Core Processor

– GPU: $4\times$ NVIDIA GeForce RTX 3090

– Mem: $8\times$ DDR4-3200 ECC RDIMM - 32GB