A Low-cost Strategic Monitoring Approach for Scalable and Interpretable
  Error Detection in Deep Neural Networks
We present a highly compact run-time monitoring approach for deep computer
vision networks that extracts selected knowledge from only a few (down to
merely two) hidden layers, yet can efficiently detect silent data corruption
originating from both hardware memory and input faults. Building on the insight
that critical faults typically manifest as peak or bulk shifts in the
activation distribution of the affected network layers, we use strategically
placed quantile markers to make accurate estimates about the anomaly of the
current inference as a whole. Importantly, the detector component itself is
kept algorithmically transparent to render the categorization of regular and
abnormal behavior interpretable to a human. Our technique achieves up to ~96%
precision and ~98% recall of detection. Compared to state-of-the-art anomaly
detection techniques, this approach requires minimal compute overhead (as
little as 0.3% with respect to non-supervised inference time) and contributes
to the explainability of the model.
Florian Geissler, Syed Qutub, Michael Paulitsch, Karthik Pattabiraman
A Low-cost Strategic Monitoring Approach for Scalable and Interpretable Error Detection in Deep Neural Networks

###### Abstract

We present a highly compact run-time monitoring approach for deep computer vision networks that extracts selected knowledge from only a few (down to merely two) hidden layers, yet can efficiently detect silent data corruption originating from both hardware memory and input faults. Building on the insight that critical faults typically manifest as peak or bulk shifts in the activation distribution of the affected network layers, we use strategically placed quantile markers to make accurate estimates about the anomaly of the current inference as a whole. Importantly, the detector component itself is kept algorithmically transparent to render the categorization of regular and abnormal behavior interpretable to a human. Our technique achieves up to \(\sim\)96% precision and \(\sim\)98% recall of detection. Compared to state-of-the-art anomaly detection techniques, this approach requires minimal compute overhead (as little as 0.3% with respect to non-supervised inference time) and contributes to the explainability of the model.

## 1 Introduction

Deep neural networks (DNNs) have reached impressive performance in computer vision problems such as object detection, making them a natural choice for problems like automated driving [1]. However, DNNs are known to be highly vulnerable to faults. For example, even small changes to the input such as adding a customized noise pattern that remains invisible to the human eye, can stimulate silent prediction errors [8]. Similarly, modifying a single out of millions of network parameters, in the form of a bit flip, is sufficient to cause severe accuracy drops [14].

Because DNNs are being deployed in safety-critical applications such as autonomous vehicles (AVs), we need efficient mechanisms to detect errors that cause such silent data corruptions (SDC). Beyond the functional part, trust in the safety of the application requires that the error detectors are interpretable by the user, so that he/she can develop an intuitive understanding of the regular
and irregular behavior of the network [2]. In an AV, for example, a user who does not trust an automated perception component due to its opaque decision-making, will not trust a black-box fault monitor either. Therefore, it is important to build interpretable error detectors for DNNs.

The goal of error detection is to supervise a small, yet representative subset of activations - during a given network inference - for comparison with a previously extracted fault-free baseline. This leads to three key challenges: **(1)** How can one compress the relevant information into efficient abstractions? **(2)** How can one efficiently perform the anomaly detection process, for complex patterns? **(3)** Can the anomaly detection decision be understandable to a human, so that insights are gained about the inner workings of the network?

Unfortunately, no existing approach satisfactorily addresses all three of the above challenges (Sec. 2). This paper presents an solution using a monitoring architecture that taps into the intermediate activations only at selected strategic points and interprets those signatures in a transparent way, see Fig. 1. Our approach is designed to detect SDC-causing errors due to input corruptions _or_ hardware faults in the underlying platform memory. Our main observation that underpins the method is that an SDC occurs when a fault _either_ increases the values of a few activations by a large margin (referred to here as an _activation peak shift_), or the values of many activations each by a small margin (_activation bulk shift_). As Fig. 2 shows, the former is observed typically for platform faults, while the latter is observed for input faults. We then use discrete quantile markers to distill the knowledge about the variation of the activation distribution in a given layer. Conceptually, within a faulty layer, we can expect a large change of only the top quantiles for activation peak shifts, and small changes of the lower and medium quantiles for bulk shifts (Fig. 2). This idea allows us to produce discriminative features for anomaly detection from a small number of monitored elements, with a single detector.

Figure 1: Monitoring architecture for quantile shift detection.
In summary, we make the following contributions in this paper:

* We demonstrate that even for complex object detection networks, we can identify anomalous behavior from quantile shifts in only a few layers.
* We identify minimal sets of relevant features and discuss their universality across models.
* We efficiently differentiate input and hardware fault classes with a single detector.
* We show that the anomaly detection process can be achieved with algorithmically transparent components, such as decision trees.

The article is structured as follows: Sec. 2 discusses related work, while Sec. 3 describes our experimental setup. We present our method in Sec. 4, and the results of our evaluation in Sec. 5.

Figure 2: **(a)** The feature map appearance is slightly changed with noise and massively affected by the memory FI. **(b)** Noise causes a small shift of multiple quantiles from the affected layer onwards (activation bulk shift). **(c)** The layer with the memory FI shows a large shift of the maximum quantile (activation peak shift), which then propagates to other quantiles.
## 2 Related Work

There are three main categories of related work.

**Image-level Techniques**: Input faults can be detected from the image itself (i.e., before network inference), in comparison with known fault-free data, resulting for example in specialized blur detectors [15]. However, these techniques do not necessarily relate to SDC in the network, as image-level corruptions may be tolerated by the model.

**Activation Patterns**: Methods to extract activation patterns range from activation vectors [5] to feature traces [24, 25]. However, these techniques do not scale well to deeper models as they result in a massive number of monitored features and large overheads. Zhao et al. [26] attempt to reduce the monitoring effort by leveraging only activations from selected layers and compressing them with customized convolution and pooling operations. This leads to a rather complex, non-interpretable detector component, and the selection of monitored layers remains empirical.

**Anomaly Detection** techniques establish clusters of regular and anomalous data to efficiently categorize new input. In single-label problems, such as image classification, fault-free clusters are typically formed by samples that belong to the same individual label [12], suggesting that those samples also share common attributes in the space of intermediate activations. This technique does not generalize to multi-label problems though, such as object detection, as many objects (in the form of bounding boxes and labels) are represented in the same image. More abstracted clustering rules such as the maximum activation range per layer have been proposed [4, 18]. However, these detectors omit more subtle errors within the activation spectrum, for example resulting from input faults. In other work [24, 25, 26], a secondary neural network is trained to perform the detection process. This comes at the cost that the detector then does not feature algorithmic transparency [2] and hence the anomaly decision is not understandable to a human. The same limitations are found in the context of detector subnetworks that are trained to identify adversarial perturbations [19].

**Summary**: We see that none of the prior techniques satisfactorily address the challenges outlined earlier. We present a new technique to overcome this problem in this paper.

## 3 Experimental Setup and Preliminary Study

**Models and Datasets:** We use the three classic object detection networks Yolo(v3), Single Shot Detector (SSD), and RetinaNet from the _open-mmlab_[3] framework, as well as the two standard image classification networks ResNet50 and AlexNet from _torchvision_[21]. Object detection networks are pretrained on Coco [20] and were retrained on Kitti [6], with the following AP50 baseline performances: Yolo+Coco: 55.5%, Yolo+Kitti: 72.8%, SSD+Coco: 52.5%, SSD+Kitti: 66.5%, RetinaNet+Coco: 59.0%, RetinaNet+Kitti: 81.8%. Image classification models were pretrained on ImageNet [17], providing accuracies of
78.0% (ResNet) and 60.0% (AlexNet) for the test setup. The data was split in a ratio of 2:1 for detector training and testing. All models are run in _Pytorch_ with the IEEE-standard FP32 data precision [16].

**Fault Modes:** Input faults are modeled using _torchvision_[21] transform functions and are applied in three different magnitudes to the raw RGB images. We select three perturbation patterns that are popular in computer vision benchmarks such as ImageNet-C [11] for our analysis: i) _Gaussian noise_ due to low lighting conditions or noise in the electronic signal of the sensor device. Low (0.1), medium (1), and high (10) noise is tested. ii) _Gaussian blur_, reflecting for example a camera lens being out of focus. We choose a kernel size of \((5,9)\) and a symmetric, variable standard deviation \((0.3,1,3)\). iii) _Contrast reductions_ simulate poor lighting conditions or foggy weather. We adjust the contrast by a factor between zero (no contrast, gray image) and one (original image). The selected models have different vulnerabilities to input faults, for example, the two image classification models ResNet and AlexNet are highly sensitive to contrast adjustments, but are rather robust to noise and blur faults. For the remaining models, the trend is reversed.

Hardware faults are modeled as single bit flips in the underlying memory and injected using _PytorchAlfi_[9]. Such flips can occur randomly either in the buffers holding temporary activation values (_neuron_ fault), or in dedicated memory which holds the parameters of the network (_weight_ faults). We group both neuron and weight faults into a single class _memory_ fault. This approach is in line with previous work [23, 24, 25, 13, 7, 18, 4]. We target all convolutional layers.

**Fault Metrics:** First, detectable uncorrectable errors (DUE) can occur when invalid symbols such as _NaN_ or _Inf_ are found among the activations at inference time. During fault injection, we observe DUE events only for memory faults, with rates \(<\)1% across all models. DUEs can be generated also at the detector stage, in the process of adding up feature map sums that contain platform errors. The rates for such events vary between 0.2% and 5.1% with our method. While DUE errors may affect the system's availability, they are considered less critical as they are readily detectable and there is no need for further activation monitoring [7].

In this article, we are concerned therefore only with silent data corruption (SDC), events that lead to a silent alteration of the predicted outcome. For image classification networks, this is represented by a change in the top-1 class prediction. For object detection systems, we use an asymmetric version of the IVMOD metric [23] as SDC criterion, i.e., an image-wise increment in the FP or FN object numbers is counted as SDC. Each experiment was done with a subset of 100 images of the test data set, running 100 random FIs on each image individually. For hardware faults, SDC rates are typically low (\(\sim\)1\(-\)3%) since drastic changes will result only from bit flips in the high exponential bits of the FP32 data type [18, 7]. Therefore, an additional 500 epochs with accelerated FI only into the three highest exponential bits are performed for both flavors of memory faults. Overall, the faulty and fault-free data is found to be balanced at a ratio of about 2:1.
## 4 Model

**Notational Remarks:** We use the range index convention, i.e., a vector is given as \(\mathbf{x}=(x_{i})=(x^{i})\), a matrix reads \(\mathbf{A}=(A_{ij})\), and similarly for higher-dimensional tensors.

**Monitoring Approach:** Let us denote a four-dimensional activation tensor that represents an intermediate state of a convolutional neural network as \(\mathbf{T}=(T_{n,c,h,w})\in\mathds{R}^{N\times C\times H\times W}\), where \(N\) is the sample number, \(C\) the number of channels, \(H\) the height, and \(W\) the width. We list \(n\) as running global sample index, where samples may be further grouped in batches. An output tensor of a specific layer \(l\in[1,\dots L]\) shall be given as \(\mathbf{T}^{l}\), with \(L\) being the total number of monitored layers of the model. Subsets of a tensor with fixed \(n,c\) are called feature maps. Our monitoring approach first performs the summation of individual feature maps and subsequently calculates quantile values over the remaining kernels, see Fig. 1,

\[\left(F_{n,c}\right)^{l} =\sum_{h,w}(T_{n,c,h,w})^{l}, \tag{1}\] \[\left(q_{n}\right)_{p}^{l} =\left(Q_{p}((F_{n,c})^{l})_{n}\right). \tag{2}\]

Here \(Q_{p}\) is the quantile function for the percentile \(p\) which acts on the \(n\)-th row of \((F_{n,c})^{l}\). In other words, \(Q_{p}\) reduces the kernel dimensions \(c\) to a set of discrete values where we use the 10-percentiles, i.e., \(p\in[0,10,20,30,\dots,90,100]\). The result is a quantile value set, \(q_{p}\), for a given image index \(n\) and layer \(l\). Note that both the summation and the quantile operations (and hence the detector) are invariant under input perturbations such as image rotations.

**Supervised Layers:** We intercept the output activations of all convolutional layers, as those layers provide the vast majority of operations in the selected computer vision DNNs. Yet, the same technique can be applied to any neural network layer.

**Reference Bound Extraction:** Applied to a separate data set \(D_{\text{bnds}}\), the above technique is used pre-runtime to extract reference bounds which represent the minimum and maximum feature sums during fault-free operation:

\[\begin{split} q_{p,\min}^{l}&=\min_{n\in D_{\text{ bnds}}}\left((q_{n})_{p}^{l}\right),\\ q_{p,\max}^{l}&=\max_{n\in D_{\text{bnds}}}\left(( q_{n})_{p}^{l}\right),\end{split} \tag{3}\]

For \(D_{\text{bnds}}\), we randomly select 20% of the training data [4].

**Anomaly Feature Extraction:** For a given input during runtime, Eqs. (1) to (2) are used to obtain the quantile markers of the current activation distribution. Those are further processed to a so-called _anomaly feature vector_ which quantifies the similarity of the observed patterns with respect to the baseline references of Eq. 3,

\[q_{p}^{l}\rightarrow\frac{1}{2}\left(f_{\text{norm}}(q_{p}^{l},q_{p,\min}^{l}, q_{p,\max}^{l})+1\right). \tag{4}\]
Here, \(f_{\rm norm}\) normalizes the monitored quantiles to a range of \((-1,1)\) by applying element-wise (\(\epsilon=10^{-8}\) is a regularization offset)

\[f_{\rm norm}(a,a_{\rm min},a_{\rm max})=\begin{cases}\tanh\left(\frac{a-a_{\rm max }}{|a_{\rm max}|+\epsilon}\right)&\text{if $a\geq a_{\rm min}$},\\ \tanh\left(\frac{a_{\rm min}-a}{|a_{\rm min}|+\epsilon}\right)&\text{if $a<a_{\rm min }$}.\end{cases} \tag{5}\]

Intuitively, the result of Eq. 5 will be positive if \(a\) is outside the defined minimum (\(a_{\rm min}\)) and maximum (\(a_{\rm max}\)) bounds (approaching \(+1\) for very large positive or negative values). The function is negative if \(a\) is within the bounds (lowest when barely above the minimum), and will become zero when \(a\) is of the order of the thresholds. In Eq. 4, a shift brings features to a range of \((0,1)\) to facilitate the interpretation of feature importance. Finally, all extracted features are unrolled into a single anomaly feature vector \(\mathbf{q}=((q^{l})_{p})=[q^{1}_{0},q^{2}_{0},\ldots q^{L}_{0},q^{1}_{10}, \ldots q^{L}_{100}]\), that will be the input to the anomaly detector component.

**Anomaly Detector:** We use a decision tree [10] approach to train an interpretable classifier, leveraging the _sklearn_ package [22]. The class weights are inversely proportionally to the number of samples in the respective class to compensate for imbalances in the training data. As a measure of the split quality of a decision node we use the Gini index [10]. To avoid overfitting of the decision tree, we perform cost-complexity pruning [22] with a factor varying between \(1\times 10^{-5}\) and \(2\times 10^{-5}\), that is optimized for the respective model.

To investigate fault class identification, we study three different detector modes with varying levels of fault class abstractions and quantify each mode \(x\in\{cls,cat,sdc\}\) by precision, \(\rm P_{x}=TP_{x}/(TP_{x}+FP_{x})\) and recall \(\rm R_{x}=TP_{x}/(TP_{x}+FN_{x})\). Here we abbreviated true positives (TP), false positives (FP), and false negatives (FN). In the class mode (\(cls\)), we consider only those detections as true positives where the predicted and actual fault modes (see Sec. 3) coincide exactly. Cases where SDC is detected correctly but the fault class does not match will be counted as either FP or FN in this setting. In the category mode (\(cat\)), those SDC detections are considered true positives where the predicted and actual fault class fall into the same category of either _memory fault_ or _input fault_\(=\{noise,blur,contrast\}\). That means, fault class confusions within a category will not reduce the performance in this mode. The final precision and recall values for the class and category mode are given as the average over all classes or categories, respectively. Finally, in the mode \(sdc\), we consider all cases as true positives where SDC was correctly identified regardless of the specific fault class. This reflects a situation where one is only interested in the presence of SDC overall, rather than the specific fault class.

## 5 Results

### Detector Performance

**Error Detection:** Tab. 1 shows the precision, recall, and decision tree complexity for the studied detectors and models. When all extracted features are
leveraged by the decision tree classifier (referred to as _full_ model), the average class-wise detection precision varies between 93.9% (ResNet) and 97.5% (RetinaNet+Kitti), while the recall is between 97.1% (RetinaNet+Coco) and 99.1% (Yolo+Kitti). If only the fault category needs to be detected correctly, we find \(\text{P}_{\text{cat}}>95\%\) and \(\text{R}_{\text{cat}}>94\%\). Correct decisions about the presence of SDC only are done with \(\text{P}_{\text{sdc}}>96\%\) and \(\text{R}_{\text{sdc}}\geq 98\%\). Across models, we observe (not shown in Tab. 1) that the most common confusion are false positive noise detections, leading to a reduced precision in the individual _noise_ class (worst case 75.8% for ResNet). The recall is most affected by memory faults (lowest individual class recall 90.6% for RetinaNet+Coco).

The detection rates of the full model in Tab. 1 outperform the ones reported in the comparable approach of Schorn et al. [25] (using feature map tracing)

\begin{table}
\begin{tabular}{c|c c c|c c c|c} \multirow{2}{*}{**Model**} & \multicolumn{3}{c|}{**P(\%)**} & \multicolumn{3}{c|}{**R(\%)**} & \multicolumn{2}{c|}{**DT**} \\ \cline{2-9}  & \(\text{P}_{\text{cls}}\) & \(\text{P}_{\text{cat}}\) & \(\text{P}_{\text{sdc}}\) & \(\text{R}_{\text{cls}}\) & \(\text{R}_{\text{cat}}\) & \(\text{R}_{\text{sdc}}\) & \(\text{N}_{\text{ft}}/\text{N}_{\text{l}}\) \\ \hline \hline
**Yolo+Coco** & & & & & & & \\ full & 95.8 & 96.4 & 96.1 & 98.2 & 98.6 & 98.4 & 825/75 \\ red (avg) & 93.3 & 94.6 & 93.4 & 97.4 & 96.3 & 96.7 & 2/2 \\ \hline
**Yolo+Kitti** & & & & & & & \\ full & 97.3 & 97.5 & 97.4 & **99.1** & 99.3 & 99.2 & 825/75 \\ red (avg) & 92.6 & 92.1 & 92.0 & 97.3 & 96.4 & 96.8 & 3/2 \\ \hline
**SSD+Coco** & & & & & & & \\ full & 96.6 & 97.2 & 96.6 & 98.2 & 98.5 & 98.3 & 429/39 \\ red (avg) & 95.2 & 96.3 & 94.9 & 96.5 & 94.5 & 95.9 & 3/3 \\ \hline
**SSD+Kitti** & & & & & & & \\ full & 96.0 & 97.1 & 96.2 & 98.4 & 98.7 & 98.6 & 429/39 \\ red (avg) & 92.8 & 94.6 & 92.1 & 98.0 & 97.7 & 98.2 & 2/2 \\ \hline
**RetinaNet+Coco** & & & & & & & \\ full & 96.6 & 95.7 & 96.9 & 97.1 & 94.9 & 98.0 & 781/71 \\ red (avg) & **96.6** & 96.6 & 96.5 & 97.0 & 94.6 & 98.2 & 2/2 \\ \hline
**RetinaNet+Kitti** & & & & & & & \\ full & **97.5** & 97.3 & 97.5 & 98.6 & 98.2 & 98.7 & 781/71 \\ red (avg) & 96.2 & 96.6 & 95.9 & **98.6** & 97.8 & 98.9 & 2/2 \\ \hline
**ResNet+Imagenet** & & & & & & & \\ full & 93.9 & **98.3** & **97.6** & 98.1 & **99.6** & **99.4** & 583/53 \\ red (avg) & 92.1 & **97.6** & **96.7** & 98.3 & **99.6** & **99.5** & 3/3 \\ \hline
**AlexNet+Imagenet** & & & & & & & & \\ full & 96.1 & **98.3** & 97.3 & 98.4 & 99.2 & 99.0 & 55/5 \\ red (avg) & 93.2 & 96.8 & 95.0 & 98.0 & 99.0 & 98.8 & 4/3 \\ \hline \end{tabular}
\end{table}
Table 1: Precision (\(P\)), Recall (\(R\)), and decision tree (DT) complexity - given as the number of used features (\(\text{N}_{\text{ft}}\)) and monitored layers (\(\text{N}_{\text{l}}\)) - for different setups. Every detector was retrained 10 times with different random seeds and the averages across all runs are given. Errors are shown when relevant to the given rounding. We list both the classifiers making use of all extracted quantiles (_full_) and the averaged reduced (_red_) detector models, where guided feature reduction was applied, see Fig. 3. Best-in-class detectors are highlighted in each column.
and the blur detection in Huang et al. [15] in terms of precision and recall. When using alternative metrics (not shown in Tab. 1) for comparison with other detector designs, we find that our method achieves class-wise misclassification rates ranging between 0.7% and 2.0%, depending on the model, which is on par with the results for example in Cheng et al. [5]. Similarly, the calculated class-wise true negative rates vary between 99.6% and 99.8%, reaching or exceeding the classifier performance in Zhao et al. [26]. Note that all mentioned references are limited to image classification networks.

**Feature Reduction:** The number of monitored features can be drastically reduced without significantly affecting the detection performance. This means that many quantiles represent similar information and further distillation can be applied. For feature reduction, we follow two steps: First, all quantile features of the full model are ranked according to their Gini importance [22] in the decision tree. Then, we retrain the classifier with a successive number of features, starting from the most important one only, to the two most important ones, etc. A reduced model is accepted as efficient if it recovers at least 95% of both the precision and recall performance of the original model with all features.

Fig. 3 shows the results of the feature reduction. Inspecting performance trends from larger to smaller feature numbers, we observe that the detection rate stagnates over most of the elimination process, before dropping abruptly when the number of used features reduces beyond a limit. On average, the number of monitored features and layers that are required to maintain close-to-original performance (as defined above) are as few as 2 to 4 and 2 to 3, respectively. For a model like Yolo, this means that only 2 out of the 75 convolution layers have to be supervised. The average characteristics of the resulting detector models is shown in Tab. 1 as reduced (_red_) model.

### Minimal Monitoring Features

**Minimal Feature Search:** The feature reduction process in Sec. 5.1 demonstrates that only few strategic monitoring markers are needed to construct an

Figure 3: Precision and recall of class-wise SDC detection when reducing the number of monitored features (average of 10 independent runs).
efficient detector model. In this section, we elaborate further to what extent the model can be compressed, and which features are the most relevant. We apply the following strategy, starting from a full classifier model using all quantile features: 1) Apply the feature reduction technique described in Sec. 5.1 to identify minimal monitoring features that maintain at least 95% of the original precision and recall. This combination of features is added to a pool of minimal model candidates. 2) A new instance of the full model is initiated and all feature candidates from the pool are eliminated. Return to the first step to find alternative candidates until a certain search depth (we choose 24) is exhausted.

**Universal Trends:** The identified minimal feature combinations are shown in Fig. 4. We find that just 2 features from 2 different layers are sufficient to constitute an efficient error detector for all studied models except for AlexNet (4 features from 3 layers).

Almost universally, one of the monitored layers needs to be among the very last layers of the deep neural network. Since memory faults are injected randomly across the network, errors in the last layers would go unnoticed otherwise. Only for SSD models, it turns out that most of the SDC faults occur in earlier layers, so that a supervision of the last layers is less crucial to achieve a similar statistical detection performance. We observe that it is favorable to supervise a higher percentile (e.g., \(q_{100}\)) in the later layers, especially in more shallow networks (AlexNet and SSD). This is because in shallow networks, peak shifts have a shorter propagation path and hence it is more important to intercept faults directly. This can only be achieved by the highest percentiles. In models with ReLU activation functions (all except Yolo here), the minimum quantile does not serve as a meaningful peak shift monitor as negative activations are clipped automatically.

A second monitoring marker should to be set in the first half of the network layer stack. This helps to identify input faults (which are interceptable from the very first layer) and discriminate them from memory faults. Either a low or high percentile can be chosen for supervision.

**Explainability:** Given the above generalizable trends and the fully transparent nature of the classifier, we can make statements about the inner workings of the DNN that correlate a given input with an anomalous or regular outcome. Those statements can be interpreted intuitively by a human as a proxy of a decision, and hence qualify as an explanation [2].

### Overhead

We measure the average inference time per image when running the supervised model on random input, using the _Torch profiler_[21]. The profiled overall self compute time in Fig. 5 is shared between CPU and GPU. Compared to the feature map tracing method of Schorn et al. [24, 25], the quantile operation introduces additional compute, but at the same time saves the time of storing large tensors, due to the compression of many feature sums into only a few quantiles. Between these two opposing trends, full quantile monitoring turns out to be _faster_ than feature map tracing for all the studied models except for the
shallow AlexNet, as shown in Fig. 5. If only selected layers are monitored to create a reduced classifier, the overhead can be decreased significantly. We find that the impact of minimal quantile monitoring on the overall inference time is between +0.3% and +1.6% for all studied object detection DNNs. For the image classification networks, on the other hand, quantile monitoring imposes a more significant overhead of +10.7% (ResNet) and +53.8% (AlexNet). This is because those networks have a much smaller number of parameters such that the relative impact of quantile extraction with respect to the total number of operations is higher. Across all models, minimal quantile monitoring is \(>10\%\) faster than feature map tracing. In absolute numbers, the respective saving in inference time can be up to \(\sim\)10ms, which is a significant improvement for applications operating at real-time, for example object detection in a self-driving vehicle.

Figure 4: Minimal combinations of features as identified by the search process in Sec. 5.2. All combinations in (a)-(e) constitute a reduced classifier model with at least 95% of the performance of the respective full model. Inset numbers designate the percentile numbers (or combinations thereof if multiple combinations are equally valid).
### Comparison with Other Detector Approaches

Alternative to a decision tree, we can deploy a linear machine learning model for error detection (similar to [24]). We study the feasibility of doing so in this section. For this setup, we select Yolo+Kitti to train a classifier for 1000 epochs using the Adam optimizer and cross entropy loss. A batch size of 100 and learning rates optimized between \(1\times 10^{-4}\) and \(5\times 10^{-3}\) were chosen. In the simplest form, with a multi-layer-perceptron, the algorithmic transparency is preserved and we find \(\mathrm{P_{cls}}=86.0\%\) and \(\mathrm{R_{cls}}=95.7\%\). If more hidden linear layers are added, higher detection rates can be achieved at the cost of explainability. For example, including one extra hidden layer with 64 neurons [24], we find a performance of \(\mathrm{P_{cls}}=88.9\%\) and \(\mathrm{R_{cls}}=96.3\%\), with three such extra layers we obtain \(\mathrm{P_{cls}}=91.7\%\) and \(\mathrm{R_{cls}}=95.1\%\). Compared to decision trees, however, this strategy suffers from more complex hyperparameter tuning and large training times. Therefore, decision trees are a better fit for our use case.

## 6 Summary and Future Work

In this paper, we show that critical silent data corruptions in computer vision DNNs (originating either from hardware memory faults or input corruptions) can be efficiently detected by monitoring the quantile shifts of the activation distributions in specific layers. In most studied cases, it is sufficient to supervise two layers with one quantile marker each to achieve high error detection rates up to \(\sim\)96% precision and \(\sim\)98% recall. We also show that the strategic monitoring location can be associated with the concept of intercepting bulk and peak activation shifts, which gives a _novel, unifying perspective on the dependability of

Figure 5: Average inference time per image accumulated over CPU and GPU. We compare the original inference, reduced and full quantile monitoring, and feature map tracing (method of [24, 25]). In the setup, we run 100 random images with a batch size of 10 (with GPU enabled) and repeat 100 independent runs. System specifications: Intel® CoreTM i9-12900K, Nvidia GeForce RTX 3090.
_DNNs_. Due to the large degree of information compression in this approach, the compute overhead of the approach is in most cases only between 0.3% and 1.6% compared to the original inference time, and outperforms the comparable state of the art. In addition, we show that the method contributes to the model's explainability as the error detection decision is interpretable and transparent. For future work, we can further guide the search for optimized minimal feature combinations, for example, by taking into account specifics of the model architecture.

**Acknowledgement:** We thank Neslihan Kose Cihangir and Yang Peng for helpful discussions. This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 956123. This work was partially funded by the Federal Ministry for Economic Affairs and Climate Action of Germany, as part of the research project Safe-Wahr (Grant Number: \(19A21026C\)), and the Natural Sciences and Engineering Research Council of Canada (NSERC).
