[1]Jan Küchler

1]ControlExpert GmbH, Marie-Curie-Straße 3, Langenfeld, 40764, Germany a]These
authors contributed equally to this work

# Uncertainty estimates for semantic segmentation: providing enhanced
reliability for automated motor claims handling

0000-0001-9087-6230<EMAIL_ADDRESS>Daniel Kröll
<EMAIL_ADDRESS>Sebastian Schoenen<EMAIL_ADDRESS>Andreas Witte<EMAIL_ADDRESS>[  [

###### Abstract

Deep neural network models for image segmentation can be a powerful tool for
the automation of motor claims handling processes in the insurance industry. A
crucial aspect is the reliability of the model outputs when facing adverse
conditions, such as low quality photos taken by claimants to document damages.
We explore the use of a meta-classification model to assess the precision of
segments predicted by a model trained for the semantic segmentation of car
body parts. Different sets of features correlated with the quality of a
segment are compared, and an AUROC score of 0.915 is achieved for
distinguishing between high- and low-quality segments. By removing low-quality
segments, the average $mI\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$ of the
segmentation output is improved by 16 percentage points and the number of
wrongly predicted segments is reduced by 77%.

###### keywords:

Semantic segmentation, Motor claims management, Meta-classification,
Uncertainty quantification, False positive detection

##  1 Introduction

In the rapidly evolving world of automotive insurance, technological
advancements are reshaping the landscape. Efficient and accurate claims
handling remain key success factors for the insurance industry. At the heart
of this process is damage assessment, traditionally reliant on manual methods.
This procedure often required experts to either make on-site visits to inspect
damaged cars or, increasingly common today, review photographs provided by
claimants. While this approach is thorough, it is also time-consuming and
vulnerable to human biases and errors.

Figure 1: Photograph of a car (left), taken to highlight issues that can
negatively affect the performance of DNN segmentation models: reflections,
dirt and bad exposure. Result of our semantic segmentation model (right),
trained to segment car body parts. Predicted segments are shown as colored
overlays. A few mistakes in the prediction are highlighted by red boxes: a
reflection on the door is segmented as a molding part, and a part of the rear
left rim is identified as an air intake.

The advent of new computer vision techniques, particularly semantic
segmentation [1], opens up possibilities to automate and streamline the damage
assessment process. By segmenting images into categorized car parts and
damages, this holds the potential to identify, classify and localize car
damages. Embracing these techniques could empower the insurance industry to
cut operational costs, expedite claim processing, and crucially, boost
accuracy.

However, any technology-driven solution requires rigorous validation of its
reliability. While deep neural networks (DNNs) have demonstrated exceptional
performance in semantic segmentation tasks [2, 3], the variability in images
of damaged cars - influenced by factors like lighting conditions, vehicle
models, capture angles, and other variables - can introduce uncertainties.
Addressing this challenge is of paramount importance.

Figure 1 shows an example for an image with some of the aforementioned issues,
together with the semantic segmentation map of car body parts. Among other
mistakes, a small area at the rim of the rear left wheel is identified as an
air intake, likely due to dirt obscuring the usual features expected for a
wheel.

To ensure a reliable and trustworthy damage assessment leveraging these
technologies, the incorporation of uncertainty estimates into semantic
segmentation is indispensable. By doing so, the industry can not only
revolutionize the damage assessment but also make it transparent, consistent,
and trustworthy, truly elevating the standards of automotive insurance claims
handling.

Various approaches have been proposed to provide a measure of uncertainty in
the model results for semantic segmentation. While the output scores of a DNN
are correlated with the accuracy of the result, models are often overconfident
and output high probabilities even for wrong results [4, 5, 6]. Methods to
estimate a model uncertainty include, for example, Monte Carlo sampling with
dropout [7]. Ensemble methods assess the uncertainty by comparing the results
of multiple, slightly different models trained for the same task [8]. Using
distillation techniques, even single models can be trained to predict the
pixel-wise uncertainty in a segmentation result [9].

In this work, we explore the use of a meta-classification [10] model to
predict the uncertainty of individual segments [11]. This approach has the
advantage of not requiring modifications to the segmentation model, nor to its
training, and has a relatively low computational overhead during inference.

As detailed in the following, uncertainty measures are first defined pixel by
pixel, based on the softmax probability output of the segmentation network
together with the loss gradient of the last convolutional layer. They are
aggregated over predicted segments, and used, together with the predicted
class of a segment and its size, to build a classification model that
distinguishes between well and wrongly predicted segments. The score of this
classifier is used as a measure of the uncertainty in the prediction. A low
uncertainty result can be automatically processed with high confidence, while
a high uncertainty score can indicate the need of human oversight. In special
cases, the uncertainty score can be used to improve the segmentation mask. By
removing segments with a high uncertainty from the segmentation mask, the
precision of the segmentation output can be improved for the cost of reducing
the recall.

##  2 Pixel- and segment-wise uncertainty measures

The output of a semantic segmentation network with a final softmax layer are
the pixel-wise probabilities $p_{i}^{k}$ for every semantic class $k=1,...,N$,
with the index $i$ running over all pixel coordinates. The predicted class for
every pixel is the one with the highest probability,
$\hat{c}_{i}=\arg\max_{k}p_{i}^{k}$.

The probability of the predicted class for a pixel,
$\hat{p}_{i}=\max_{k}p_{i}^{k}$ quantifies the confidence in the result [12],
thus $1-\hat{p}_{i}$ is used as one measure of the pixel-wise uncertainty.

Following Ref. [11], two further quantities are defined, measuring the
dispersion of the pixel-wise probabilities:

  * •

the entropy

$E_{i}=\frac{1}{\log K}\sum_{k=1}^{N}p_{i}^{k}\log(p_{i}^{k})\ ,$

  * •

as well as the difference between the two largest softmax values,

$D_{i}=\hat{p}_{i}-\max_{k\neq\hat{c}_{i}}p_{i}^{k}\ ,$

which targets cases where the network predicts a similar probability for the
two most likely classes.

Ref. [13] introduces a gradient-based approach for uncertainty quantification
in semantic segmentation. The gradient of a categorical cross entropy loss
with respect to the last convolutional layer of the segmentation network can
be computed efficiently. When taking the predicted class $\hat{c_{i}}$ as the
one-hot label per pixel, these gradients quantify the likeness of the result
to the examples in the training data set. Intuitively, larger gradients mean
that the weights of the convolutional layer need to be changed more strongly
to accommodate the input, therefore indicating an uncertain result. As
proposed in [13], the partial norm of the pixel-wise gradients is taken as an
additional measure of the uncertainty.

Figure 2 shows qualitative heat-maps of the pixel-wise uncertainty measures
for the example image of Figure 1. Due to the labeling accuracy, the
boundaries between segments of different classes are uncertain and highlighted
in the heat-maps. The wrongly predicted segments at the door and at the rim of
the rear left wheel are also indicated by high pixel-wise uncertainties. On
the other hand, the uncertainties vary strongly in these segments. The pixel-
wise uncertainties are aggregated to segment-wise measures, in order to build
features for the classification of high- and low-quality segments.

Figure 2: Qualitative heat-maps of $1-\hat{p}_{i}$ (top left), $1-D_{i}$ (top
right), the entropy $E_{i}$ (bottom left) and the gradient uncertainty (bottom
right), for the example image shown in Figure 1. Darker shades indicate higher
pixel-wise uncertainties.

The predicted semantic segmentation mask for an image is split into a set
$\hat{\mathcal{K}}$ of segments, i.e. connected areas of the same class. The
pixel-wise uncertainty measures are aggregated over these segments by
averaging the individual values over the full segment. The values are also
averaged separately over the boundary and the inner part of the segment, as
defined by Ref. [11], because the boundaries typically exhibit higher
uncertainties. Additionally, the standard deviation of the pixel-wise
uncertainty distributions on the boundary, inner and full segment is used as
an input to the meta-classification model.

(a) Sketch of a segmentation result.

false
---
negative
area
true
---
positive
area
false positive area
---
$I\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$$I\mkern-2.0muo\mkern
1.0muU\mkern-1.0mu_{\mathrm{adj.}}$$p$ (b) Relevant areas for quality metrics.

Figure 3: Sketch of a segmentation result and the quality metrics for one of
the segments. (3(a)) A ground truth segment of class _A_ (black dashed
rectangle) is covered by three predicted segments: two of class _A_ (blue),
divided by a segment of a different class _B_ (red). The correctly segmented
area is indicated by the two blue shaded rectangles.
(3(b)) The $I\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$ of the left-most
predicted segment is small, as it is calculated by dividing the blue shaded
area by the intersection of the ground truth and the predicted segment,
respectively. In contrast, for the $I\mkern-2.0muo\mkern
1.0muU\mkern-1.0mu_{\mathrm{adj.}}$ the area covered by the other segment of
class  _A_ is disregarded. For the precision, $p$, the correctly predicted
area is compared only to the full predicted segment.

The quality of segments is defined with respect to the ground truth using the
measure of intersection over union [14]. The ground truth segmentation mask is
split into a set $\mathcal{K}$ of segments, analogously to the prediction.
Predicted segments are then compared to all ground truth segments with a
matching class label and a non-trivial intersection, denoted as
$\left.\mathcal{K}\right|_{\hat{k}}$. For a predicted segment
$\hat{k}\in\hat{\mathcal{K}}$ and the union of the matching and intersecting
ground truth segments $K=\bigcup_{k\in\left.\mathcal{K}\right|_{\hat{k}}}k$,
the segment-wise intersection over union is defined as

$I\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu(\hat{k})=\frac{\left|\hat{k}\cap
K\right|}{\left|\hat{k}\cup K\right|}\ .$

Figure 3 shows a sketch to clarify the definition of the $I\mkern-2.0muo\mkern
1.0muU\mkern-1.0mu$ and further quality metrics, which are defined and
motivated below.

Rottmann et al. [11] define the adjusted intersection over union,
$I\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu_{\mathrm{adj.}}$, by restricting the
denominator to the union of the predicted segment with the area of the
matching ground truth segments which is not covered by other predicted
segments of the same class. This makes the metric more forgiving in cases
where, e.g., one ground truth segment is covered by two disjoint predicted
segments, which are split by a small, wrongly predicted area. Intuitively,
both predicted segments describe a fraction of the ground truth segment well,
even though, in the original definition, the $I\mkern-2.0muo\mkern
1.0muU\mkern-1.0mu$ is small.

In a similar fashion, we assess the quality of predicted segments by their
precision,

$p(\hat{k})=\frac{\left|\hat{k}\cap K\right|}{\left|\hat{k}\right|}\ ,$

i.e., the fraction of pixels in the predicted segment which overlap with a
matching ground truth segment. For completely wrong predictions, i.e. without
overlap of the predicted segment and the ground truth, $p=I\mkern-2.0muo\mkern
1.0muU\mkern-1.0mu=I\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu_{\mathrm{adj.}}=0$.
Only for at least partially correct segments, the behavior of the metrics
differ and $p\geq I\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu_{\mathrm{adj.}}\geq
I\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$. By choosing the precision instead of
the $I\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$, we intentionally neglect to
quantify how much of the ground truth segment is covered. For some downstream
tasks using the segmentation information of a partial, but precise segment can
still be valuable. As an example, a damage detected on a precise but
incomplete segment of a car body part is, in many cases, sufficient to provide
a correct cost calculation.

##  3 Segment quality classification

The aforementioned metrics are used to train a segment meta-classifier for a
semantic segmentation model for car body parts. The segmentation model is a
fully convolutional DNN, distinguishing between 70 car body parts. Segment
metrics and ground truth information are collected for about 3000 labeled
images, which were used as a validation data set for the training of the
segmentation model. An independent set of about 1000 labeled images, which was
not used for the training of the segmentation model, provides a test set of
segments with ground truth information.

Segments with $p>0.5$ are labeled as correctly predicted. The threshold is
determined from the distribution of the segment precision, c.f. Figure 4,
visual investigation of segments with varying precision and in consideration
of downstream tasks. The performance of the meta-classification model does not
strongly depend on the chosen precision threshold, as will be detailed below.

Figure 4: Distribution of the segment-wise precision. Segments with $p>0.5$
are selected as correct predictions. The population of segments at very low
precision consists mostly of small, wrongly predicted segments.

Various classification models are trained to predict the binary segment
quality, i.e. classify $p>0.5$ versus $p\leq 0.5$, and the resulting
performance is compared. Different sets of features are tested, as listed in
Table 1.

Table 1: List of segment-wise features included in the three feature sets: ‘all’, ‘reduced’, and ‘uncertainty only’. Features | all | reduced | uncertainty only
---|---|---|---
Averages of pixel-wise uncertainties | ✓ | ✓ | ✓
Relative segment size | ✓ | ✓ |
Predicted class | ✓ | ✓ |
Standard deviation of pixel-wise uncertainties | ✓ |  |
Boundary/inner pixel information | ✓ |  |

Two types of classifiers are tested: a gradient boosted decision tree, based
on the XGBoost library [15] as a high performance method [16], as well as a
linear regression classifier [17], as a simpler baseline. The XGBoost hyper-
parameters are optimized in a grid search employing 5-fold cross validation on
the training data set.

Table 2 lists the area under the receiver operator characteristic curves
(AUROC, [18]) obtained for all combinations of classifier types and segment
feature sets. The precision-recall curves are displayed in Figure 5. The
XGBoost model trained using all features performs best, achieving an AUROC
score of $91.6\%\pm 0.2\%$ and an average precision of $93.4\%\pm 0.2\%$.
Reducing the feature set by excluding the standard deviation of the
uncertainty distributions and split of segment features into boundaries and
inner areas only entails a minor decrease in performance. The achieved AUROC
score is $91.5\%\pm 0.2\%$ with an average precision of $93.3\%\pm 0.2\%$. The
results are comparable to the classification results achieved in [11] for a
different model and data set.

The predicted class and the segment size are important for the performance of
the XGBoost classifier. Without them, the AUROC score is reduced to $89.0\%\pm
0.3\%$ and is on par with the results obtained using the simpler logistic
regression of the input features.

Table 2: AUROC scores for all evaluated combinations of classifier types and feature sets, with statistical uncertainties due to the size of the test data set. Feature set | XGBoost | Log. reg.
---|---|---
all | 0.916 $\pm$ 0.002 | 0.891 $\pm$ 0.003
reduced | 0.915 $\pm$ 0.002 | 0.885 $\pm$ 0.003
uncertainty only | 0.890 $\pm$ 0.003 | 0.882 $\pm$ 0.003
Figure 5:  Precision in dependence of the recall obtainable for selecting low-
quality segments for all evaluated combinations of classifier types and
feature sets. The legend states the average precision $\mathrm{AP}$.

For further studies, the XGBoost model trained with the reduced feature set is
used. The output of this meta-classification model is scaled to a range of
$[0,1]$ with higher values for segments with a low predicted quality. It is
used as a measure of the uncertainty for a segment and is strongly correlated
with the segment precision ($\rho=0.74$), and the two variants of
$I\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$ ($\rho\geq 0.88$). This correlation
prevails even when choosing a different segment precision threshold to define
the binary target for meta-classification.

The uncertainty measure can be used to remove low-quality segments from the
predicted mask. This prevents downstream tasks from including wrong
predictions, which can lead to false positive results for car body parts that
are not at the predicted location or not even displayed in an image. The
failure modes of the segmentation model include small, wrongly predicted
segments within larger areas of correct predictions. This can be caused by
reflections or dirt on the surface of the car. Segments with an uncertainty
larger than a specific threshold are removed from the segmentation mask. If
such a segment is fully enclosed by just one other segment, i.e. if all
neighboring pixels have the same predicted class in the original prediction,
it is replaced by the enclosing class. Otherwise, the segment is set to the
“background" class, thus preventing downstream tasks from using the pixels for
further results. Figure 6 shows an example of the segment-wise uncertainties
and the corrected segmentation mask for the image shown in Figure 1. The
wrongly detected air intake segment at the rim is removed, preventing wrong
input to subsequent processes. The wrongly predicted molding segment on the
door is removed, and replaced by the surrounding door class. The segment-wise
uncertainty map provides comprehensive and easy-to-use information about the
reliability of each segment for further applications. For example, if damages
are found only on segments with a low uncertainty, the claims handling process
can be automated with high confidence in the end result. Individual high
uncertainty segments can be removed from the segmentation mask, in order to
improve the quality of the result.

Figure 6: Heat-map of the segment-wise uncertainties (left) and corrected
segmentation mask (right) for the example image shown in Figure 1. Within the
heat-map, darker shades indicate higher uncertainties and the colored contours
show segments with an uncertainty above the threshold which are either removed
and set to the background class (red), or replaced by the unambiguous
surrounding class (green), as decided by the algorithm described in the text.

The quality of a segmentation mask for an image can be characterized by the
mean (i.e., class averaged) $I\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$. Given
the sets of predicted classes, $\hat{\mathcal{C}}$, and of the classes in the
ground truth labels, $\mathcal{C}$, for an image, this metric is defined as

$mI\mkern-2.0muo\mkern
1.0muU\mkern-1.0mu=\frac{1}{\left|\hat{\mathcal{C}}\cup\mathcal{C}\right|}\sum_{c\in\hat{\mathcal{C}}\cup\mathcal{C}}\frac{tp_{c}}{tp_{c}+fp_{c}+fn_{c}}\
,$

where $tp_{c}$, $fp_{c}$, and $fn_{c}$ are the numbers of true positive, false
positive and false negative predicted pixels of class $c$, respectively.
Notably, any class which is neither in the prediction nor in the labels does
not affect the $mI\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$, while classes which
are in the predicted segments but not in the ground truth labels (and vice
versa) reduce the $mI\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$ of a segmentation
mask.

Figure 7: Distribution of the $mI\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$
difference, $\Delta$, between the corrected and the original mask. The red,
hatched area marks entries with $\Delta<0$, indicating a quality degradation,
occurring only for $f_{\Delta<0}=2.6\%$ of the images. The inset shows a
scatter plot of the $mI\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$ of the
corrected prediction in dependence on the $mI\mkern-2.0muo\mkern
1.0muU\mkern-1.0mu$ of the original mask.

The $mI\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$ is computed image by image for
the original segmentation mask, as well as for the corrected mask, to quantify
the impact of removing segments with a high uncertainty. Figure 7 shows the
distribution of the difference between these two values. On average over all
images, the $mI\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$ is improved by
$\overline{\Delta mI\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu}=0.16$,
corresponding to an increase of the average $mI\mkern-2.0muo\mkern
1.0muU\mkern-1.0mu$ from $0.50$ to $0.66$. The standard deviation of the
distribution of $\Delta mI\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$ is $0.09$ on
the test set. For $>97\%$ of the images in the test set an improvement of the
result is observed. In the rare cases that the correction procedure results in
a $mI\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$ decrease, usually small,
irregularly formed but precise segments within the larger area of a
misidentified car body part are removed. Figure 7 also shows the
$mI\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$ values for corrected masks in
dependence on the uncorrected result. Improvements can be seen over a large
range of $mI\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$.

A major factor of the improvement is the removal of small segments, in turn
leading to a wrongly predicted class being removed from the mask entirely.
Even though only a small fraction of pixels in the image is affected, the
effect on the $mI\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu$ is significant
because every class has the same weight. The number of wrongly predicted
classes per image is reduced from 6.3 to 1.4, on average, with standard
deviations of 4.0 and 1.5, respectively. At the same time, the a small
decrease in the number of correctly predicted classes is observed as well,
reducing the number from 11.2 to 10.6, with standard deviations of 7.3 and
6.9. Crucially, this reduction prevents false positive detections in
downstream tasks.

##  4 Conclusion

In this work, the development and application of a meta-classification model
is presented, which is used to assess the quality of the output of a semantic
segmentation model for car body parts. Pixel-wise uncertainties are derived
from the softmax probabilities and gradients, and are combined to segment-wise
features. A gradient boosted decision tree classifier based on the average
uncertainty features per segment has been trained to distinguish between
precise and imprecise segments. The resulting meta-model achieves an AUROC
score of $0.915\pm 0.002$. The outputs of this classifier provide a
comprehensive uncertainty measure for each segment.

In a production setting, the meta-classification model runs as a post-
processing step after evaluating the car body part segmentation model. The
resulting uncertainty scores are then used to remove low-quality segments from
the predictions. This removal prevents false positive detections in downstream
tasks and improves the segmentation mask quality for this use-case by
$\overline{\Delta mI\mkern-2.0muo\mkern 1.0muU\mkern-1.0mu}=0.16$.

The proposed method can improve the reliability of a segmentation model
output. In the context of motor claims handling, this has been proven to be a
valuable tool for the automation of damage assessment tasks.

## Statements and Declarations

The authors declare that they have no known competing financial interests or
personal relationships that could have appeared to influence the work reported
in this paper.

## Acknowledgments

We thank Hanno Gottschalk, Matthias Rottmann, Svenja Uhlemeyer and the IZMD at
Bergische Universität Wuppertal for helpful discussions and useful advice.

## References

  *   * Shotton et al. [2006] Shotton, J., Winn, J., Rother, C., Criminisi, A.: TextonBoost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) Computer Vision – ECCV 2006, pp. 1–15. Springer, Berlin, Heidelberg (2006). https://doi.org/10.1007/11744023_1
  * Wang et al. [2021] Wang, J., Sun, K., Cheng, T., Jiang, B., Deng, C., Zhao, Y., Liu, D., Mu, Y., Tan, M., Wang, X., Liu, W., Xiao, B.: Deep high-resolution representation learning for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 43(10), 3349–3364 (2021) https://doi.org/10.1109/TPAMI.2020.2983686
  * Yu et al. [2023] Yu, Y., Wang, C., Fu, Q., Kou, R., Huang, F., Yang, B., Yang, T., Gao, M.: Techniques and challenges of image segmentation: A review. Electronics 12(5) (2023) https://doi.org/10.3390/electronics12051199
  * Mehrtash et al. [2020] Mehrtash, A., Wells, W.M., Tempany, C.M., Abolmaesumi, P., Kapur, T.: Confidence calibration and predictive uncertainty estimation for deep medical image segmentation. IEEE Transactions on Medical Imaging 39(12), 3868–3878 (2020) https://doi.org/10.1109/tmi.2020.3006437 1911.13273
  * Hein et al. [2019] Hein, M., Andriushchenko, M., Bitterwolf, J.: Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, pp. 41–50 (2019). https://doi.org/10.1109/CVPR.2019.00013
  * Guo et al. [2017] Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q.: On calibration of modern neural networks. In: Proceedings of the 34th International Conference on Machine Learning - Volume 70. ICML’17, pp. 1321–1330. JMLR.org, Sydney, NSW, Australia (2017)
  * Kendall et al. [2017] Kendall, A., Badrinarayananm, V., Cipolla, R.: Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding. In: Kim, T.-K., Zafeiriou, S., Brostow, G., Mikolajczyk, K. (eds.) Proceedings of the British Machine Vision Conference (BMVC), pp. 1–12. BMVA Press, London, United Kingdom (2017). https://doi.org/10.5244/C.31.57
  * Lee et al. [2020] Lee, H.J., Kim, S.T., Lee, H., Navab, N., Ro, Y.M.: Efficient Ensemble Model Generation for Uncertainty Estimation with Bayesian Approximation in Segmentation (2020)
  * Holder and Shafique [2021] Holder, C.J., Shafique, M.: Efficient uncertainty estimation in semantic segmentation via distillation. In: 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pp. 3080–3087 (2021). https://doi.org/10.1109/ICCVW54120.2021.00343
  * Lin and Hauptmann [2003] Lin, W.-H., Hauptmann, A.: Meta-classification: Combining multimodal classifiers. In: Zaïane, O.R., Simoff, S.J., Djeraba, C. (eds.) Mining Multimedia and Complex Data, pp. 217–231. Springer, Berlin, Heidelberg (2003). https://doi.org/10.1007/978-3-540-39666-6_14
  * Rottmann et al. [2020] Rottmann, M., Colling, P., Paul Hack, T., Chan, R., Huger, F., Schlicht, P., Gottschalk, H.: Prediction error meta classification in semantic segmentation: Detection via aggregated dispersion measures of softmax probabilities. In: 2020 International Joint Conference on Neural Networks (IJCNN). IEEE, Glasgow, United Kingdom (2020). https://doi.org/10.1109/IJCNN48605.2020.9206659
  * Hendrycks and Gimpel [2017] Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. In: 5th International Conference on Learning Representations, ICLR 2017\. OpenReview.net, Toulon, France (2017). https://openreview.net/forum?id=Hkg4TI9xl
  * Maag and Riedlinger [2023] Maag, K., Riedlinger, T.: Pixel-wise Gradient Uncertainty for Convolutional Neural Networks applied to Out-of-Distribution Segmentation (2023). https://doi.org/10.48550/arXiv.2303.06920
  * Jaccard [1912] Jaccard, P.: The distribution of the flora in the alpine zone. The New Phytologist 11(2), 37–50 (1912) https://doi.org/10.1111/j.1469-8137.1912.tb05611.x
  * Chen and Guestrin [2016] Chen, T., Guestrin, C.: XGBoost. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, USA (2016). https://doi.org/10.1145/2939672.2939785
  * Shwartz-Ziv and Armon [2022] Shwartz-Ziv, R., Armon, A.: Tabular data: Deep learning is not all you need. Information Fusion 81, 84–90 (2022) https://doi.org/10.1016/j.inffus.2021.11.011
  * Pedregosa et al. [2011] Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 12, 2825–2830 (2011) 1201.0490
  * Davis and Goadrich [2006] Davis, J., Goadrich, M.: The relationship between precision-recall and ROC curves. In: Proceedings of the 23rd International Conference on Machine Learning. ICML ’06, pp. 233–240. Association for Computing Machinery, New York, NY, USA (2006). https://doi.org/10.1145/1143844.1143874


# Selecting Related Knowledge via Efficient Channel Attention for Online
Continual Learning

Ya-nan Han  Jian-wei Liu  Department of Automation, College of Information
Science and Engineering, China University of Petroleum , Beijing, Beijing,
China

###### Abstract

Continual learning aims to learn a sequence of tasks by leveraging the
knowledge acquired in the past in an online-learning manner while being able
to perform well on all previous tasks, this ability is crucial to the
artificial intelligence (AI) system, hence continual learning is more suitable
for most real-word and complex applicative scenarios compared to the
traditional learning pattern. However, the current models usually learn a
generic representation base on the class label on each task and an effective
strategy is selected to avoid catastrophic forgetting. We postulate that
selecting the related and useful parts only from the knowledge obtained to
perform each task is more effective than utilizing the whole knowledge. Based
on this fact, in this paper we propose a new framework, named Selecting
Related Knowledge for Online Continual Learning (SRKOCL), which incorporates
an additional efficient channel attention mechanism to pick the particular
related knowledge for every task. Our model also combines experience replay
and knowledge distillation to circumvent the catastrophic forgetting. Finally,
extensive experiments are conducted on different benchmarks and the
competitive experimental results demonstrate that our proposed SRKOCL is a
promised approach against the state-of-the-art.

###### keywords:

Continual learning, related knowledge, Catastrophic forgetting, Knowledge
distillation

††journal: arXiv

##  1 Introduction

While deep learning techniques have realized tremendous successes in a wide
range of applications over the last decade, most approaches mainly focus on
single, isolated problems where the model need to be retrained or redesigned
from scratch when they face a new task. As a result, when working in non-
stationary environment where stream of instances involves different learning
task, then a traditional deep neural network will face catastrophic forgetting
of previously learned knowledge [1]. The aim of continual learning is to learn
consecutive tasks by leveraging the knowledge obtained in the past while
should not forget how to perform previously learned tasks.

In the current literature, most of exist continual learning methods are
limited to this situation where the whole task data are observed at each step
and the agent would learn the current task by many epochs. However, such
learning methods are poorly scalable for most real-word scenarios, where data
arrives in a stream style and the agent need to rapidly learn new skills from
the current task. In this work, we would develop our continual learning
approach in online scenario where every task would be learned in an online
manner with training data coming sequentially. Optimizing DNN with such a
regime is more practical and appealing and requires many training episode and
various techniques in this process while struggling when data comes in a
single epoch setting [2, 3].

In general, in order to effectively learn in online continual learning setup,
the model should be able to avoid catastrophic forgetting of previously
learned information and can perform well on the all tasks at the end of
learning. While the initial successes have been achieved recent years, such as
regularization-based methods [4, 5, 6], architecture-based methods [7, 8],
etc, there are still a lot of open problems that need to be addressed.
Regularization-based continual learning methods compute the important weight
parameters for every task and control their change when facing the new task.
However, these kinds of method have limited ability to overcome catastrophic
forgetting, especially in online setting. In contrast, architecture-based
methods avoid catastrophic forgetting by increasing the capacity of neural
network. Obviously, these methods are not well suitable for a large number of
tasks since they need to a large number of memories for every task to
alleviate the forgetting [9].

In addition, a favor latent representation can effectively improve the
performance of the model especially in online setting. Unfortunately, most of
these methods learn a generic representation by using the whole knowledge.
Instead of this, we consider that learning to pick only the relevant
beneficial knowledge from the prior knowledge to perform current task instead
of using all the previously learned knowledge is more reasonable. This is
inspired by human learning. For example, a student from computer science major
with a mathematical background should be more preferable to learn the
prerequisite courses such as data structure, database system, digital signal
processing, etc. This basic mathematical knowledge is can help to learn
professional knowledge where one selects the useful information from
everyone’s mathematical background according to the context of each course.

Inspired from above discussion, we propose a framework, called Selecting
Related Knowledge for Online Continual Learning (SRKOCL). The SRKOCL can learn
multiple tasks sequentially with neural network while not forgetting the
knowledge obtained from the old tasks via experience replay strategy and pool
knowledge distillation strategy.

More specifically, jointly training via episodic memory is employed to avoid
catastrophic forgetting. However, during the jointly training with previous
data stored from previous tasks, the imbalance between the previous and
current data is another crucial challenge, which can cause a bias towards
current task. To overcome this challenge, our SRKOCL uses pooled outputs
distillation [10] technology to avoid this, which can match global statistics
at various feature levels between the previous and current models to further
enforce a suitable trade-off between the old and new knowledge. In addition,
we propose to apply an efficient channel attention mechanism to automatically
select the most relevant representation from the generic feature bank to
improve the classification accuracy. Finally, experimental comparisons are
introduced to assess the proposed SRKOCL approach.

To sum up, our main contributions in this work are listed as follows:

1) We proposed a novel SRKOCL framework in online continual learning scenario
where this framework can learn a sequence of task while not forgetting the old
knowledge learned from old tasks.

2) Consider that only part of the relevant knowledge is useful for the current
task, so we propose to apply an efficient channel attention mechanism to learn
to automatically pick the useful knowledge to perform current task.

3) To this end, we perform comprehensive experiments on several commonly used
datasets in continual learning literature to validate the effectiveness of our
proposed SRKOCL algorithm against a series of state-of-the-art algorithms from
continual learning, and the comparative experiment results illustrate that our
SRKOCL algorithm is a promising approach for continual learning.

##  2 Related Work

Continual learning, also called lifelong learning, is a long standing research
direction in the field of machine learning, which considers learning multiple
task sequentially where the agent has to learn the new knowledge from the
current task while avoiding to forget previous knowledge learned from past
tasks [9]. Roughly speaking, the existing works for continual learning can be
broadly divided into three categories: regularization-based, architecture-
based and memory-based.

In order to avoid catastrophic forgetting, the regularization-based approaches
such as Elastic Weight Consolidation(EWC)[4], Synaptic Intelligence(SI) [5],
etc, typically attempt to employ a regularization term to the objective
function and then preserve as much as possible the weights that are important
to perform previous tasks. The differences among these works lie in the weight
importance measurement. In other relevant work [6, 11, 12], regularization is
applied to penalize the feature space on task learned in the past to preserve
the knowledge in the original model. While these works can obtain a good
solution for whole tasks, such methods learn a generic representation only
using class label for each task and ignore the fact that there is part of
useful knowledge is to the current task.

These types of architecture-based methods attempt to change architectural
properties in response to new knowledge, and then the old information on
already learned task is maintained by this way. For example, Rusu et al.[8]
develop progressive neural which dynamically expand the architecture by
assigning new sub-module with fix capacity to be learned on new tasks and
freezing the modules trained on past tasks. Unfortunately, this strategy that
grows the network to avoid forgetting in continual learning setting would be
not scalable for a large scale tasks due to their computational cost and a
fixed capacity that the assumption is unrealistic [9].

These memory-based approaches alleviate catastrophic forgetting by storing
previous experience explicitly while the old instance stored in memory would
be used for rehearsal. For instance, early researchers [13] try to alleviate
forgetting of previously knowledge using experience memory which is replayed
regularly, and such approaches are still utilized today [11]. In other works,
episodic memory is applied as a constraint to mitigate catastrophic forgetting
while allowing beneficial transfer of knowledge to past tasks, such as GEM
[14], Rwalk [15].

##  3 Problem Formulation

Following the recent continual learning literature [14, 3], for the online
continual learning setting, a model needs to learn new tasks sequentially from
an online data stream while each instance is observed only once. Classically,
a deep model $f:R{{\kern 1.0pt}^{\;d}}\,\to\;\,R{{\kern 1.0pt}^{\;c}}$,
parameterized by a vector $\theta\;$, is utilized. When encountering the input
instance $x\;_{i}^{t}$ from the task $t$, we should make the formula
${f_{\theta}}\left({x\,_{i}^{t}}\right)\approx y\,_{i}^{t}$ holds for each
instance. Furthermore, we adopt the multi-head leaning setup, that means, the
classifier has to access to task identity. The model aims to train classifier
to learn sequential tasks from data stream while not forgetting the knowledge
obtained from previous tasks.

##  4 The Proposed SRKOCL Framework

The final goal of continual learning is to mimic human learning. Catastrophic
forgetting is the long-standing challenge in continual learning, so the
starting point for this setting to achieve this goal is to overcome
catastrophic forgetting, that means, the model should keep the previous
knowledge learned from old tasks when learning new knowledge from current
task. In addition, the model should learn to pick the related knowledge from
the previously obtained knowledge to complete the learning of current task. In
this section, we describe our proposed SRKOCL approach in detail that aim to
address these two problems. Fig. 1 shows our proposed SRKOCL framework, and
this model is mainly composed of three parts: experience replay, pooled
outputs distillation and efficient channel attention mechanism. More
concretely, experience replay via episodic memory is used to overcome
catastrophic forgetting, and pooled outputs distillation is applied to prevent
the data imbalance between the current and the previous task during jointly
training. Then, we apply an efficient channel attention mechanism to
automatically pick the useful and related knowledge to improve the performance
of the model. The details of each phase are illustrated in the following
paragraphs.

Experience Replay (ER).Compare to the other types of methods, such as the
regularization-based approaches, etc, the memory-based approaches show great
advantages. Recently, ER has been widely applied to avoid catastrophic
forgetting in continual learning task. For ER strategy we utilize, we conduct
two simple modifications. Firstly, a subset of the instances from prior tasks
is stored by a memory buffer ${\cal M}$ with the size of $M$, where we select
ring buffer as the writing strategy which can ensure the balance between all
classes of each task. Then, when the incoming minibatch ${B_{n}}$ is observed,
we concatenate ${B_{n}}$ with another minibatch ${B_{\cal M}}$ of instances
obtained from the memory buffer ${\cal M}$. After that, an SGD step with
combined batch is performed to update our model. Finally, we select cross
entropy as our loss function and the training objective can be formulated as
following:

${L_{pre}}\left({\theta,B_{n}^{t}\cup
B_{m}^{er}}\right)=-\mathbb{E}_{\left({{x_{\kern 1.0pt}}_{i}^{k},\;{y_{\kern
1.0pt}}_{i}^{k}}\right){\kern 1.0pt}\,\cup\,{{\cal M}_{\kern
1.0pt}}^{1\;:\;k}}\sum\limits_{c=1}^{C}{\;{{\bf{1}}_{\left[{c\;=\;{y_{\kern
1.0pt}}_{i}^{k}}\right]}}}\log\left({\sigma\left({{f_{\theta}}\left({{x_{\kern
1.0pt}}_{i}^{k}}\right)}\right)}\right.{\rm{}}$ (1)

Figure 1: The illustration of the novel SRKOCL network. Above (a): shows
SRKOCL at training time. When the task t arrives, the latent representation is
learned via a feature extractor (such as ResNet) and here denotes loss
function. To prevent forgetting, the neural network is retained via experience
replay with an episodic memory which is written by ring buffer. Right
bottom(c): is the feature extractor which is used to learn the latent
representation. Left bottom (b): denotes the efficient channel attention
module added after each convolutional layer which can select the useful and
relevant knowledge instead of taking the whole knowledge.

Pooled Outputs Distillation (POD).Consider that in the training process the
current and previous task data are used to joint training simultaneously.
However, compared to the current task data, previous task data are limited in
this step, which would lead to an imbalance between the current and the
previous task data. Such imbalance will make this model more favorable to
learn the current task, leading to performance degradation. Therefore, in this
work we apply POD technology [10] to remove the imbalance and further make a
trade-off between the new and old knowledge.

Specifically, POD technology consists in minimizing the difference of global
statistics at various feature levels between the new and old model. Let $z$
denote an embedding tensor of size $H\times W\times C$ and then, We extract a
POD embedding $\Phi$, which consists of two complementary intermediate
statics: the $W\times C$ height-pooled slices and the $H\times C$ width-pooled
slices of the embedding tensor $z$:

$\Phi\left(z\right)=\left[{\frac{1}{W}\sum\limits_{w=1}^{W}{z\left[{:,w,:}\right]\left\|{\frac{1}{H}\sum\limits_{h=1}^{H}{z\left[{h,:,:}\right]}}\right.}}\right]\in{R^{\left({H+W}\right)\times
C}}$ (2)

where $\left[{\cdot\;\left\|{\;\cdot}\right.}\right]$ denotes concatenation
over the channel axis, and then the differences for both the previous and
current model are computed at several layers( in fact, each stage of a ResNet
[16]). Then, we selected L2 distance between the two sets of embedding as the
POD loss function:

${L_{pod}}\left(\theta\right)=\frac{1}{L}\sum\limits_{l=1}^{L}{\;{{\left\|{\Phi\left({f_{l}^{t}}\right)-\Phi\left({f_{l}^{t-1}}\right)}\right\|}^{2}}}$
(3)

Efficient Channel Attention Mechanism.In general, the learned previous
knowledge is shared among all tasks. So, the model should enjoy the ability
that can automatically pick the useful information from the generic feature
bank [17]. To address this problem, we proposed to apply efficient channel
attention [18] to make the network to recalibrate the useful and relevant
knowledge based on the input instances. In particular, an attention block is
employed after each convolutional layer and the role of this block is to
recalibrate adaptively the convolutional channels. After that, this model can
learn to boost the useful informative features according to the input. Let
$z=\left\\{{{z_{1}},{z_{2}},\cdots,{z_{c}}}\right\\}\in{R^{H\times W\times
C}}$ denotes the convolutional outputs and $H$, $W$ and $C$ denote height,
width and depth of the feature maps. The vector $s$ of size $C$ denotes the
output of every attention block, which can recalibrate the feature map
${z_{i}}\in{R^{H\times W}}$ for each channel.

${z_{i}^{{}^{\prime}}}={z_{i}}\circ{s_{i}}$ (4)

where ${s_{i}}$ denotes the scalar value for channel $i$ and $\circ$ denotes a
channel-wise multiplication.

Note that channel and its weight need a direct correspondence while avoiding
dimensionality reduction is more important than consideration of nonlinear
channel dependencies [18]. In order to capture local cross-channel interaction
and ensure both efficiency and effectiveness, the weight of ${z_{i}}$ is
computed by only considering interaction between ${z_{i}}$ and its $k$
neighbors:

${s_{i}}=\sigma\left({\sum\limits_{j=1}^{k}{w_{i}^{j}z_{i}^{j}}}\right),\;z_{i}^{j}\in\Omega_{i}^{k}$
(5)

Where $\Omega\;_{i}^{k}$ denotes the set of $k$ adjacent channels of
${z_{i}}$. Making the entire channels to share the same parameters is more
efficient, as is shown in Fig.1, that means:

${s_{i}}=\sigma\left({\sum\limits_{j=1}^{k}{{w^{j}}z_{i}^{j}}}\right),\;z_{i}^{j}\in\Omega_{i}^{k}$
(6)

This process can be achieved by a fast 1D convolution with kernel size $k$,

$s=\sigma\left({{\rm{C1D}}\left(z\right)}\right)$ (7)

where C1D denotes 1D convolution. It is noted that the kernel size of 1D
convolution need to be determined. Inspired by group convolutions [19], high-
dimensional(low-dimensional) channels should involve long-range(short-range)
convolutions when the number of groups is given. So, it is reasonable that the
kernel size $k$ has proportional to channel dimension $C$, that means, there
might be a mapping $C=\phi\left(k\right)$. In addition, consider that we
usually set the channel dimension $C$ to power of 2. Therefore, based on this
fact, we extend the simple linear function to a non-linear situation,

$C=\phi\left(k\right)={2^{\left({\lambda\cdot k+b}\right)}}$ (8)

After that, the kernel size $k$ can be automatically determined by the
following Eq.(9) when the channel dimension $C$ is known,

$k={\left|{\frac{{{{\log}_{2}}\left(C\right)}}{\lambda}+\frac{b}{\lambda}}\right|_{odd}}$
(9)

Noted that in this work the values of $\lambda$ and $b$ are set to 2 and 1
respectively. Clearly, by computing the Eq.(9), we can achieve that the longer
range interaction is involved when the channel dimension is higher while low-
dimensional channels have shorter range interaction.

##  5 Formulating Whole Model

In this section, we would illustrate our whole model. Our model incorporates a
classical convolutional network (in fact, a reduced ResNet) as a feature
extractor, and a classifier is used to make predictions. Then, we introduce
three parts to our SRKOCL model: (1) the experience replay is applied to
prevent the catastrophic forgetting; (2) the knowledge distillation
${L_{pod}}$ is to make an trade-off between the current knowledge and the
previous knowledge to remove the imbalance between the current task and the
previous task data; (3) we devise an efficient channel attention mechanism to
adaptively select the relevant knowledge from the old knowledge instead of
using the entire knowledge. Finally, the ultimate loss is formulated as
follows:

${L_{total}}={L_{pre}}+{L_{pod}}$ (10)

The objective function can be divided into two parts: the cross-entropy loss
and the pooled output distillation. Our proposed SRKOCL algorithm is
illustrated in Algorithm 1 in details.

0: The model parameters $\theta$, episodic memory ${{\cal M}_{\kern
1.0pt}}\leftarrow\left\\{{}\right\\}$ and learning Rate: $\eta$

10: memory update strategy for ${{\cal M}_{\kern 1.0pt}}$

2for  _$k=0$ to $T$_ do

3 _The train dataset ${D_{k}}$ comes sequentially_

4 for  _$n=0$ to ${N_{batches}}$_ do

5

6  _Observe a mini-batch of training instances ${B_{n}}$ from the current task
$k$_

7  _Compute the total loss ${L_{\;total}}$ using
$\left({x,y}\right)\in{B_{n}}\cup{B_{\cal M}}_{\kern 1.0pt}$ by Eq.(10)_

_8_ _ _ _Update the model parameters $\theta\leftarrow\theta-\eta{\kern
1.0pt}\Delta{\kern 1.0pt}{\kern 1.0pt}{L_{\;total}}$_

9

10  _ ${{\cal M}_{\kern 1.0pt}}\leftarrow
Update\\_Memory\left({mem\\_size,\;k,\;{{\cal M}_{\kern
1.0pt}},\;{B_{n}}}\right)$ _

11

12 return $\theta,{\kern 1.0pt}\;{{\cal M}_{\kern 1.0pt}}$

Algorithm 1 SRKOCL algorithm

##  6 Experiments

In this section, firstly, we review the benchmark datasets used in our
evaluation as well as baselines we compared against. Next, we describe the
implementation details. Finally, we perform a set of experiments to validate
the efficacy of our SRKOCL, and then, we report and analysis the experimental
results obtained on different baseline datasets.

###  6.1 Datasets and Baselines

Datasets. The following benchmarks are used to evaluate our proposed SRKOCL
algorithm. Split CIFAR100 is a variant of the CIFAR100 dataset [20], where
every task consists of 5 various classes without any replacement from the
total of 100 classes. We split CIFAR100 dataset into 20 disjoint subsets, and
each of which can be regarded as a separated task, that means in this dataset
there are 20 tasks in total; Split CIFAR10 divide the CIFAR-10 dataset [20]
into 5 various tasks and 2 classes included in every task, similarly as Split
CIFAR100; Split SVHN [21] includes 600,000 digit images from Goodle Street
View and we also divide this dataset into 5 disjoint tasks and 2 classes
contained in each task.

Baselines. We compare our SRKOCL algorithm to the following state-of-the-art
approaches: LwF (Learning without Forgetting) [6], EWC(Elastic Weight
Consolidation) [4], ICARL (Incremental Classifier and Representation Learning)
[11], GEM (Gradient Episodic Memory) [14], ER (Experience Replay) [22], FTML
(Following the Meta Learner) [23].

###  6.2 Implementation Datails

We follow the implementation details proposed in [14, 3] in all of our
experiments. Specifically, we use a reduced ResNet18 in our all experiments.
In addition, we apply stochastic gradient decent (SGD) to optimize our model
with the bath-size 10 for all approaches in online learning fashion. We use
the experience replay strategy to avoid catastrophic forgetting and the size
of episodic memory is set to 65 and 25 for every task on CIFAR and SVHN
datasets respectively. For each model, we conduct experiments five times and
the average numerical results are reported in this paper.

###  6.3 Results and Analysis

In this subsection, we perform a variety of experiments to validate the
effectiveness of SRKOCL in continual learning scenario. In the first set of
experiments, we compare the performance of our SRKOCL algorithm to the state-
of-the-art approaches on different benchmark datasets in continual learning
literatures via three evaluation metrics: average accuracy(ACC), forgetting
measure (FM) [14], learning accuracy(LA) [24]. The average accuracy denotes
the classify performance on all of tasks when the learner finishes training on
the last task $T$. Forgetting is used to evaluate the model’s ability to
maintain previous knowledge when the new knowledge is obtained, that means,
the smaller this metric, the less forgetting. Lastly, learning accuracy is
applied to measure the model’s ability to learn quickly.

Next, we offer more insight for our SRKOCL and conduct an ablation study to
evaluate the contribution of each component in this algorithm on different
datasets. The final numerical results are reported in Table 1. Here, SRKOCL-
Base: the model is trained without efficient channel attention mechanism and
feature knowledge distillation, SRKOCL-POD: the model is trained using feature
knowledge distillation alone, SRKOCL: the model is trained using feature
knowledge distillation and efficient channel attention mechanism.

Table 1: Experimental results of different algorithms on different datasets Method | Split CIFAR10(5T)
---|---
ACC($\uparrow$) | FM($\downarrow$) | LA($\uparrow$)
LWF | 0.7574$\pm$0.0216 | 0.1627$\pm$0.0308 | 0.8875$\pm$0.0186
EWC | 0.7398$\pm$0.0496 | 0.1873$\pm$0.0510 | 0.8794$\pm$0.0338
ICARL | 0.8057$\pm$0.0249 | 0.1181$\pm$0.0313 | 0.9002$\pm$0.0036
GEM | 0.8491$\pm$0.0121 | 0.0585$\pm$0.0208 | 0.8940$\pm$0.0093
ER | 0.8465$\pm$0.0174 | 0.0600$\pm$0.0151 | 0.8945$\pm$0.0121
FTML | 0.8400$\pm$0.0291 | 0.0779$\pm$0.0446 | 0.8998$\pm$0.0155
SRKOCL-Base | 0.8537$\pm$0.0109 | 0.0619$\pm$0.0163 | 0.9025$\pm$0.0095
SRKOCL-POD | 0.8726$\pm$0.0195 | 0.0365$\pm$0.0238 | 0.8919$\pm$0.0145
SRKOCL | 0.8876$\pm$0.0078 | 0.0320$\pm$0.0053 | 0.9035$\pm$0.0124
Method | Split SVHN(5T)
---|---
ACC($\uparrow$) | FM($\downarrow$) | LA($\uparrow$)
LWF | 0.8327$\pm$0.0196 | 0.1337$\pm$0.0365 | 0.9397$\pm$0.0111
EWC | 0.8265$\pm$0.0179 | 0.1466$\pm$0.0410 | 0.9439$\pm$0.0160
ICARL | 0.8689$\pm$0.0204 | 0.1062$\pm$0.0354 | 0.9539$\pm$0.0104
GEM | 0.9000$\pm$0.0159 | 0.0714$\pm$0.0308 | 0.9572$\pm$0.0102
ER | 0.9157$\pm$0.0137 | 0.0608$\pm$0.0182 | 0.9643$\pm$0.0041
FTML | 0.9330$\pm$0.0063 | 0.0422$\pm$0.0103 | 0.9668$\pm$0.0021
SRKOCL-Base | 0.9215$\pm$0.0167 | 0.0511$\pm$0.0256 | 0.9616$\pm$0.0051
SRKOCL-POD | 0.9396$\pm$0.0047 | 0.0216$\pm$0.0068 | 0.9529$\pm$0.0081
SRKOCL | 0.9410$\pm$0.0048 | 0.0169$\pm$0.0116 | 0.9527$\pm$0.0073
Method | Split CIFAR100(10T)
---|---
ACC($\uparrow$) | FM($\downarrow$) | LA($\uparrow$)
LWF | 0.4224$\pm$0.0152 | 0.2240$\pm$0.0128 | 0.6325$\pm$0.0077
EWC | 0.3824$\pm$0.0440 | 0.2686$\pm$0.0473 | 0.6338$\pm$0.0092
ICARL | 0.4648$\pm$0.0061 | 0.1877$\pm$0.0212 | 0.6401$\pm$0.0199
GEM | 0.6198$\pm$0.0135 | 0.0570$\pm$0.0104 | 0.6534$\pm$0.0166
ER | 0.6322$\pm$0.0142 | 0.0582$\pm$0.0150 | 0.6687$\pm$0.0084
FTML | 0.6272$\pm$0.0150 | 0.0850$\pm$0.0144 | 0.7023$\pm$0.0048
SRKOCL-Base | 0.6320$\pm$0.0058 | 0.0652$\pm$0.0076 | 0.6727$\pm$0.0088
SRKOCL-POD | 0.6485$\pm$0.0130 | 0.0474$\pm$0.0136 | 0.6716$\pm$0.0066
SRKOCL | 0.6698$\pm$0.0066 | 0.0490$\pm$0.0117 | 0.6945$\pm$0.0147

Table1 shows that our SRKOCL enjoys competitive performance on different
datasets implementing different evaluation criteria. Firstly, notably, our
SRKOCL outperforms other methods considered in our experiments in terms of the
final average accuracy. Secondly, SRKOCL-POD have lower ACC than SRKOCL, which
means the attention module would be beneficial to generate the favorable
representation. That means, selecting appropriate features instead of taking
all features is more reasonable. Furthermore, it is not surprised that SRKOCL-
POD is better than SRKOCL-Base in terms of overall ACC and FM with the help of
POD module since this mechanism can further alleviate catastrophic forgetting.

Compared to the state-of-the-art methods, we can make several observations.
Firstly, traditional continual learning algorithms, such as LwF, EWC, achieve
relatively poor performance on almost all datasets. Then, as a class
incremental learner, ICARL model makes an improvement via a nearest-example
algorithm with higher learning accuracies; however, this model has limited
ability to fight against catastrophic forgetting. Furthermore, ER approach
performs similarly or better than GEM algorithm in terms of ACC and FM. It is
noteworthy that both of them outperform the traditional methods on different
evaluation metrics since they use an episodic memory to remove forgetting.
While FTML can realize the ability to learn quickly when new task arrives via
meta-learning method with high LA values, this model neglects the ability of
maintaining previous knowledge, which result in lower overall performance in
contrast to our SRKOCL approach. Additionally, the above methods only capture
a generic latent representation for each task which cannot learn to select the
relevant and useful features only for the inputs of current task.

In our SRKOCL framework, an episodic memory is used to effectively avoid the
catastrophic forgetting and the knowledge distillation technology is applied
to make knowledge integration between the old and new knowledge which can
remove the bias. Notably, our SRKOCL method outperforms all baselines
considered in this paper in terms of overall ACC, confirming our discussion
earlier. The results also illustrate that the SRKOCL is a promising approach
for online continual learning setting.

##  7 Conclusions and Future Work

In this work, we highlight two main challenges in the field of online
continual learning: catastrophic forgetting and selection of relevant
knowledge. To deal with former, we use experiment replay and POD knowledge
distillation strategy, where experiment replay via an episodic memory can
effectively alleviate catastrophic forgetting and POD module can further make
a suitable trade-off between the learning of new knowledge and the keeping of
old knowledge, ultimately, further remove the forgetting due to the imbalance
of the data between the current task and previous task. In addition, we use
the efficient channel attention module to learn to select the appropriate
knowledge according to the input from the prior knowledge instead of taking
the all knowledge to further improve the performance of the model. Finally,
the comprehensive experiments are performed on different benchmark datasets
and our approach consistently realize promising results, and the result of
ablation studies on various datasets shows the contribution of each component
about the SRKOCL framework in its overall performance.

There are several directions for future work. For example, meta-learning aims
to deal with learning to learn which can leverage the past knowledge learned
from the previous task to accelerate adaptation to a new task, so in the
future we can use meta learning technology to further improve knowledge
transfer. In this paper, we have shown the effectiveness of SRKOCL in image
classification area and we can consider experiments are conducted on other
application domains, such as semantic segmentation, semantic detection and so
on. In addition, we would also consider the online learning problem for domain
incremental setting that is more complex continual learning scenario.

## References

  * [1] R. M. French, Catastrophic forgetting in connectionist networks, Encyclopedia of cognitive science (2006).
  * [2] D. SAHOO, H. Q. PHAM, J. LU, S. C. HOI, Online deep learning: Learning deep neural networks on the fly.(2018), in: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence IJCAI 2018, July 13-19, Stockholm, pp. 2660–2666.
  * [3] Q. Pham, D. Sahoo, C. Liu, S. C. Hoi, Bilevel continual learning, arXiv e-prints (2019) 1–13.
  * [4] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al., Overcoming catastrophic forgetting in neural networks, Proceedings of the national academy of sciences 114 (13) (2017) 3521–3526.
  * [5] F. Zenke, B. Poole, S. Ganguli, Continual learning through synaptic intelligence, in: International Conference on Machine Learning, PMLR, 2017, pp. 3987–3995.
  * [6] B. Leibe, J. Matas, N. Sebe, M. Welling, Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV, Vol. 9908, Springer, 2016.
  * [7] J. YOON, E. YANG, J. LEE, et al., Lifelong learning with dynamically expandable networks [j/ol] (2018).
  * [8] A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, R. Hadsell, Progressive neural networks, arXiv preprint arXiv:1606.04671 (2016).
  * [9] G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, S. Wermter, Continual lifelong learning with neural networks: A review, Neural Networks 113 (2019) 54–71.
  * [10] A. Douillard, M. Cord, C. Ollion, T. Robert, E. Valle, Podnet: Pooled outputs distillation for small-tasks incremental learning, in: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, Springer, 2020, pp. 86–102.
  * [11] S.-A. Rebuffi, A. Kolesnikov, G. Sperl, C. H. Lampert, icarl: Incremental classifier and representation learning, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, 2017, pp. 5533–5542.
  * [12] J. He, R. Mao, Z. Shao, F. Zhu, Incremental learning in online scenario, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 13926–13935.
  * [13] A. Robins, Catastrophic forgetting in neural networks: the role of rehearsal mechanisms, in: Proceedings 1993 The First New Zealand International Two-Stream Conference on Artificial Neural Networks and Expert Systems, IEEE, 1993, pp. 65–68.
  * [14] D. Lopez-Paz, M. Ranzato, Gradient episodic memory for continual learning, Advances in neural information processing systems 30 (2017) 6467–6476.
  * [15] A. Chaudhry, P. K. Dokania, T. Ajanthan, P. H. Torr, Riemannian walk for incremental learning: Understanding forgetting and intransigence, in: European Conference on Computer Vision, Springer, 2018, pp. 556–572.
  * [16] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  * [17] G. Sokar, D. C. Mocanu, M. Pechenizkiy, Self-attention meta-learner for continual learning, in: Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, 2021, pp. 1658–1660.
  * [18] Q. Wang, B. Wu, P. Zhu, P. Li, W. Zuo, Q. Hu, Eca-net: Efficient channel attention for deep convolutional neural networks, in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, 2020, pp. 11531–11539.
  * [19] Y. Ioannou, D. Robertson, R. Cipolla, A. Criminisi, Deep roots: Improving cnn efficiency with hierarchical filter groups, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2017, pp. 5977–5986.
  * [20] A. Krizhevsky, G. Hinton, et al., Learning multiple layers of features from tiny images (2009).
  * [21] J. Rajasegaran, S. Khan, M. Hayat, F. S. Khan, M. Shah, itaml: An incremental task-agnostic meta-learning approach, in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2020, pp. 13585–13594.
  * [22] A. Chaudhry, M. Rohrbach, M. Elhoseiny, T. Ajanthan, P. K. Dokania, P. H. Torr, M. Ranzato, Continual learning with tiny episodic memories (2019).
  * [23] C. Finn, A. Rajeswaran, S. Kakade, S. Levine, Online meta-learning, in: International Conference on Machine Learning, PMLR, 2019, pp. 1920–1930.
  * [24] M. Riemer, I. Cases, R. Ajemian, M. Liu, I. Rish, Y. Tu, G. Tesauro, Learning to learn without forgetting by maximizing transfer and minimizing interference, arXiv preprint arXiv:1810.11910 (2018).