# Coalitional Bayesian Autoencoders: Towards explainable unsupervised deep
learning

Bang Xiang Yong, Alexandra Brintrup  Institute for Manufacturing, University
of Cambridge, UK

###### Abstract

This paper aims to improve the explainability of Autoencoder’s (AE)
predictions by proposing two explanation methods based on the mean and
epistemic uncertainty of log-likelihood estimate, which naturally arise from
the probabilistic formulation of the AE called Bayesian Autoencoders (BAE). To
quantitatively evaluate the performance of explanation methods, we test them
in sensor network applications, and propose three metrics based on covariate
shift of sensors: (1) G-mean of Spearman drift coefficients, (2) G-mean of
sensitivity-specificity of explanation ranking and (3) sensor explanation
quality index ($SEQI$) which combines the two aforementioned metrics.
Surprisingly, we find that explanations of BAE’s predictions suffer from high
correlation resulting in misleading explanations. To alleviate this, a
”Coalitional BAE” is proposed, which is inspired by agent-based system theory.
Our comprehensive experiments on publicly available condition monitoring
datasets demonstrate the improved quality of explanations using the
Coalitional BAE.

###### keywords:

Explainable deep learning, Bayesian Autoencoders, condition monitoring,
Industry 4.0, Internet of Things

††journal: Journal of Applied Soft Computing

##  1 Introduction

With recent advances in artificial intelligence and machine learning for
Industrial Internet of Things (IIoT) devices, deep neural network (DNN) offers
a promising way to analyse large amount of data for a range of applications.
However, due to its black-box nature, DNN does not offer explainability out-
of-the-box; for instance, given the model’s prediction of the equipment’s
health in Fig. 1(a), one is unable to tell which sensors are contributing to
the prediction without additional modelling effort to obtain explanations
shown in Fig. 1(b). Being explainable is crucial to improving the trust and
safety in DNN especially for a high stake industrial applications.

Figure 1: Illustrative examples of (a) health indicator of an equipment as the
output of a DNN and (b) the local attribution scores for each sensor. At any
degradation state (0 = healthy, 1 = total failure), the higher the sensor
attribution score, the higher the importance of the sensor is to the model
making the prediction.

One way of obtaining explanations for DNN predictions is by using posthoc
explanation methods which use a separate model to assign attribution scores to
the features. Nonetheless, a critical question remains - how good are these
explanations? Recent studies in supervised DNN have shown the fragility of
such explanations [1]; despite the DNN achieving a high predictive accuracy,
the explanations to these predictions may not be reliable.

Extant explanation methods are mostly studied within the context of supervised
DNN, and their success on unsupervised DNN is even more unclear. It is
important to consider explainability for unsupervised DNNs too, as they have
many applications in the field of manufacturing, in particular when
considering unlabelled sensor data [2]. Without careful evaluation of the
explanation methods for unsupervised DNN, manufacturers are exposed to the
risk of misleading explanations when adopting these methods.

Therefore, in this paper, we focus our analysis on a manufacturing use-case:
that of ”sensor attribution scores” as a form of explanation obtained using
Autoencoder (AE) which is a type of unsupervised DNN. Moreover, we adopt the
probabilistic formulation of the AE called Bayesian Autoencoder (BAE) [3, 4]
which provides a sound theoretical foundation for analysis. As a specific use-
case, we study their applications in industrial equipment health indicator
(HI). In particular, our contributions are as follows,

  1. 1.

Formulation of two sensor attribution methods for BAE. Under the independent
likelihood assumption, we propose two methods for assigning attribution scores
to sensor inputs which are based on (1) the mean log-likelihood estimate and
(2) the epistemic uncertainty of log-likelihood estimate. We argue that the
BAE predictions are naturally decomposable into the attribution of features,
without necessitating any posthoc explanation methods.

  2. 2.

Development of ”Coalitional BAE”. We propose the Coalitional BAE which is
inspired by coalitional agent-based system theory to improve the explanation
quality. In this setup, an independent BAE is trained for every sensor, as
opposed to the conventional training of a Centralised BAE which combines all
sensor inputs under a single BAE.

  3. 3.

Quantitative evaluation metrics for explainable HI based on covariate shift of
sensors. Considering that a HI is attributable to the sensors which are either
shifting or non-shifting from the training distribution, we propose three
metrics to evaluate how well do the explanations reflect the ground truth: (1)
G-mean of Spearman drift coefficients ($G_{SDC}$), (2) G-mean of specificity-
sensitivity of explanation ranking ($G_{SSER}$) and (3) sensor explanation
quality index ($SEQI$) which is a weighted sum of the two aforementioned
metrics.

  4. 4.

Finding of misleading explanations due to correlation in outputs. Through our
experiments, we find that the quality of explanation obtained with the
Centralised BAE is poor due to correlation in the outputs. This sheds light on
the propensity of Centralised BAE to give misleading explanations which was
not noticed in extant studies. In contrast, by enforcing independence in the
model outputs, the quality of explanation attained by Coalitional BAE
consistently outperforms that of Centralised BAE.

##  2 Related work

###  2.1 Health Indicator with Autoencoders

In supervised DNN, the model learns a direct mapping from the inputs to the
target labels, whereas in unsupervised learning, the model does not have
access to the target labels and the model is trained only on the input data to
infer the possible target labels. AE is a popular unsupervised DNN, and there
exists two main schemes of obtaining a HI in the literature,

  1. 1.

AE as an automatic feature extractor. In this mode, the AE is trained to
reconstruct the input data and subsequently a supervised model or clustering
algorithm is trained to map the encoded features of lower dimensions to the
target labels [5]. Various modifications were proposed to improve the encoded
features, such as stacked sparse AEs [6, 7], enhanced regularisation method
[8] and ensemble of stacked AEs [9]. Xu et al. [10] improved the AE
classification performance by applying Gath-Geva clustering algorithm on the
encoded features.

  2. 2.

Anomaly detection. In condition monitoring applications, AEs are trained only
to reconstruct the sensor measurements during the equipment’s healthy phase.
During test phase, measurements from a degrading equipment are assumed to
deviate from the healthy training distribution and as such, would be assigned
a higher reconstruction loss. In separate studies, the reconstruction loss has
been used to differentiate healthy and damaged states of a bridge [11] and
surface-mounted devices [12]. Amarbayasgalan et al. [13] also studied the use
of reconstruction loss in time-series anomaly detection. Recently, Yong et al.
[3] proposed a new variant of AE called Bayesian Autoencoder (BAE), and
studied the use of uncertainty and reconstruction loss to detect various types
of drifts in a hydraulic system.

In this work, we investigate the explainability of HI obtained with the BAE
trained under the anomaly detection mode and build upon the work of Yong et
al. [3] on using BAE.

###  2.2 Explainable Deep Learning

A recent review by Zhang et al. [14] highlights the importance of
explainability of DNN as a future direction for its application in industry.
To explain predictions of DNN, extant methods assign importance scores to the
input features. Ribeiro et al. [15] proposed a model-agnostic method called
Local Interpretable Model-agnostic Explanations (LIME) which uses a model of
lower complexity to compute interpretable approximation of the original model.
There are also DNN-specific explanation models such as DeepLift [16] and
IntegratedGradients [17] which leverages backpropagation to assign importance
scores to the features based on the activation of the neurons.

Although these methods were designed for supervised learning, Antwarg et al.
[18] studied the use of Shapley Additive Explanation (SHAP) [19] and LIME, to
interpret the contribution of input features to the AE’s reconstruction loss.
In contrast, Bergmann et al. [20] proposed to directly use the reconstruction
loss pertaining every input feature as the feature attribution score.
Martinez-Garcia et al. [21] proposed an entropy-based method to provide visual
interpretations of the AE’s reconstructed signals in turbine health
monitoring. Liznerski et al. [22] developed Fully Convolutional Data
Description (FCDD) as a method to provide explanations to deep one-class
classifier in a semi-supervised setting.

One challenge in explainable deep learning is in evaluating the explanation
methods and there has been a lack of consensus in extant studies since
explainability is a subjective concept. In assessing these methods,
qualitative evaluation is usually adopted by visualising the saliency map [21,
22]. However, it is also important to consider quantitative methods which
enable more rigorous evaluation and to improve the explanation methods
objectively. For this reason, Yeh et al. [23] proposed the notion of fidelity
and sensitivity as performance metrics of explanation methods. In addition,
Hooker et al. [24] presented a framework for Benchmarking Attribution Methods
(BAM) with a priori knowledge of relative feature importance in the dataset
which serves as ground truth. To the best of our knowledge, however, there has
not been any comprehensive studies which use quantitative metrics to evaluate
the explanation methods for unsupervised deep learning.

##  3 Methods

###  3.1 Bayesian Autoencoder (BAE)

This section describes the notations necessary to formulate the AE from a
Bayesian perspective and the methods for training and predictions.

####  3.1.1 Notation

Assume that on a piece of industrial equipment, we have a set of $K$ sensors,
$S_{K}=\\{{s}_{k}\\}_{k=1}^{K}$. Let training data
$X=\\{\textbf{x}_{n}\\}_{n=1}^{N}$ , $\textbf{x}_{n}\in\rm I\\!R^{K\times{D}}$
where $D$ is the number of raw measurements or preprocessed features for the
$K$ sensors. We further denote $x_{k,d}$ as an entry of data point
$\textbf{x}_{n}$ for the $k$-th sensor and the $d$-th feature.

The AE is a neural network parameterised by $\theta$ which maps $X$ into a set
of reconstructed signals with the same dimensions as the input data point,
$\hat{X}=\\{\hat{\textbf{x}}_{n}\\}_{n=1}^{N}$, $\hat{\textbf{x}_{n}}\in\rm
I\\!R^{K\times{D}}$. The AE consists of two parts: an encoder
$f_{\text{encoder}}$, for mapping input data x to the latent features of lower
dimensional space, and a decoder $f_{\text{decoder}}$ for mapping the latent
features to a reconstructed signal of the input $\hat{\textbf{x}}$ (i.e.
$\hat{\textbf{x}}={f_{\theta}}(\textbf{x})=f_{\text{decoder}}(f_{\text{encoder}}(\textbf{x}))$)
[25].

####  3.1.2 Training the BAE

Bayes’ rule can be applied to the parameters of the AE, to create a BAE,

$p(\theta|X)=\frac{p(X|\theta)\ p(\theta)}{p(X)}\\\ ,$ (1)

where $p(X|\theta)$ is the likelihood and $p(\theta)$ is the prior
distribution of the AE parameters. Following Yong et al. [4], we use the
isotropic Gaussian distribution with fixed variance of 1 to have independent
likelihoods and a non-informative prior.

Since Eq. 1 is analytically intractable for a DNN which is highly non-linear
and consists of a large number of parameters, we employ the state-of-the-art
randomised maximum a posteriori (MAP) sampling method, which was developed
recently by Pearce et al. [26] called anchored ensembling for obtaining
approximate samples from the posterior distribution. One major advantage of
this method is the lower number of posterior samples ($\geq{5}$), as compared
to Markov Chain Monte Carlo (MCMC) and mean field variational inference [27]
which typically require thousands of samples. In addition, it was shown
empirically that ensembling DNNs typically yields good quality of predictive
uncertainty [28, 29] and the anchored ensembling grounds this technique in
Bayesian theory [26].

During initialisation, we independently sample a set of anchored weights
$\\{\theta_{anc,m}\\}_{m=1}^{M}$ for the ensemble of $M$ AEs from the Kaiming
uniform distribution [30] and these anchored weights remain fixed throughout
the training procedure. For each $m$-th AE in the ensemble, we minimise the
following loss function using optimisers for DNN such as Adam [31],

$Loss_{m}=\frac{1}{N}\sum^{N}_{n=1}{||\textbf{x}_{n}-f_{\theta_{m}}(\textbf{x}_{n})||^{2}}+\frac{\lambda}{N}{||\theta_{m}-\theta_{anc,m}||^{2}}$
(2)

where $\lambda$ is set as a hyperparameter to scale the prior term for
regularisation. Importantly, we note that minimising the mean-squared error
loss which is often used as the AE reconstruction loss, is equivalent to
maximising the log-likelihood of a isotropic Gaussian distribution with a
fixed variance of 1. As a result of the training procedure, we obtain a set of
approximate samples from the posterior distribution
$\\{{\hat{\theta{}_{m}}}\\}_{m=1}^{M}$.

####  3.1.3 Predicting with BAE

During the test phase, we use the ensemble to obtain $m$-samples of the
negative log-likelihood for a test data point
$\\{\scalebox{0.75}[1.0]{$-$}{\log{p({\textbf{x}_{n}^{*}}|\hat{\theta_{m})}}\\}}_{m=1}^{M}$
and subsequently compute the mean and epistemic uncertainty of the negative
log-likelihood estimate with respect to the posterior distribution,

$\operatorname*{\mathbb{E}}{}_{\theta}{[\scalebox{0.75}[1.0]{$-$}{\log{p(\textbf{x}_{n}^{*}|\theta)}}]}\approx{}\frac{1}{M}\sum_{m=1}^{M}{\scalebox{0.75}[1.0]{$-$}{\log{p({\textbf{x}_{n}^{*}}|\hat{\theta_{m})}}}}$
(3)
$\text{Var}_{\theta}[\scalebox{0.75}[1.0]{$-$}{\log{p(\textbf{x}_{n}^{*}|\theta)}}]\approx{}\frac{1}{M}\sum_{m=1}^{M}{(\scalebox{0.75}[1.0]{$-$}{\log{p({\textbf{x}_{n}^{*}}|\hat{\theta_{m})}}}-\operatorname*{\mathbb{E}}{}_{\theta}{[\scalebox{0.75}[1.0]{$-$}{\log{p(\textbf{x}_{n}^{*}|\theta)}}]}})^{2}$
(4)

Both of these measures can be used for measuring dataset shifts. For brevity,
we shall refer to
$\operatorname*{\mathbb{E}}{}_{\theta}{[\scalebox{0.75}[1.0]{$-$}{\log{p(\textbf{x}_{n}^{*}|\theta)}}]}$
and
$\text{Var}_{\theta}[\scalebox{0.75}[1.0]{$-$}{\log{p(\textbf{x}_{n}^{*}|\theta)}}]$
as $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ and
$\mathrm{Var_{\theta}(NLL)}$ respectively. Intuitively, the higher the
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ or
$\mathrm{Var_{\theta}(NLL)}$, the greater the shift from the training
distribution and the more severe the outcome [3].

####  3.1.4 Sensor Attribution Scores

The task of explaining a BAE prediction can be viewed as assigning attribution
scores to $K$ sensors, given the test data point $\textbf{x}_{n}^{*}$. We
denote the explanation method
$\Phi{(\textbf{x}_{n}^{*}{;}\>\theta{})}\mathrel{\mathop{\mathchar
58\relax}}\rm I\\!R^{K\times{D}}\to\rm I\\!R^{K}$ and the $k$-th sensor
attribution score $\phi_{n,k}\in\Phi{(\textbf{x}_{n}^{*}{;}\>\theta{})}$.

We postulate that the BAE is naturally equipped with explanation methods to
obtain these attribution scores and does not necessitate the use of posthoc
explanation methods. Under the independent likelihood assumption, the overall
log-likelihood is the sum of the log-likelihoods due to $D$ features and $K$
sensors,

$\log{p(\textbf{x}^{*}_{n}|\theta{})}=\sum_{k=1}^{K}\sum_{d=1}^{D}{\log{p(x^{*}_{n,k,d}|\theta{})}}$
(5)

Next, we note that the mean (variance) of a sum of independent random
variables is simply the sum of the mean (variance) of the independent random
variables. Thereafter, it is straightforward to decompose
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ and
$\mathrm{Var_{\theta}(NLL)}$ into a sum of their components, resulting in two
types of explanation methods,

$\Phi^{\operatorname*{\mathbb{E}}{}_{\theta}}{(\textbf{x}_{n}^{*}{;}\>\theta{})}=[\sum_{d=1}^{D}{\operatorname*{\mathbb{E}}{}_{\theta}{[\scalebox{0.75}[1.0]{$-$}{\log{p(x^{*}_{n,k,d}|\theta{})}]}}}]_{k=1}^{K}$
(6)
$\Phi^{\text{Var}_{\theta}}{(\textbf{x}_{n}^{*}{;}\>\theta{})}=[\sum_{d=1}^{D}{\text{Var}_{\theta}{[\scalebox{0.75}[1.0]{$-$}{\log{p(x^{*}_{n,k,d}|\theta{})}]}}}]_{k=1}^{K}$
(7)

The $\phi_{n,k}^{\operatorname*{\mathbb{E}}{}_{\theta}}$ and
$\phi_{n,k}^{\text{Var}_{\theta}}$ are local explanations for each BAE
prediction which have higher granularity than global explanations attained
through conventional feature selection methods [32] which assign importance to
features based on the entire dataset. One way to obtain such global
explanations from local explanations is by averaging the sensor attribution
scores over the dataset
$[\>\frac{1}{N}\sum_{n=1}^{N}{\phi_{n,k}}\>]_{k=1}^{K}$.

Furthermore, we emphasise our analysis on the sensor level which is closer to
application, although we can also decompose the HI into the lower feature
level. That is, one would expect an operator to be more capable of acting upon
the sensors rather than the measurements themselves.

###  3.2 Centralised and Coalitional BAE

In a usual setup of BAE, one would concatenate multiple sensor data streams
and forward pass to a single BAE model on the entire data, which we call
Centralised BAE (Fig. 2(a)). This creates a dependency on all input sensors
when computing the attribution score for one sensor (e.g sensor attribution
for $s_{1}$ is dependent on input data from all sensors
$s_{1},s_{2},...,s_{K}$). As we will find in our empirical results in Section
5.1, this configuration induces correlation in the BAE outputs and
subsequently harms the quality of explanation.

(a) Centralised BAE

(b) Coalitional BAE

Figure 2: Data flow for the configurations of (a) Centralised BAE and (b)
Coalitional BAE with $K$ sensors where each sensor measures $D$ features. The
number of BAE samples (size of ensemble) is $M$.

There are two possible sources for the correlation in BAE outputs: (1)
correlation in input data and (2) model parameters and architecture. For (1),
the correlation in outputs which arise from the correlation in input sensors
is actually doing the right thing - explaining the correct state of the
correlated sensors.

Conversely, we posit it is mainly due to (2), that the correlation in BAE
outputs are detrimental to the quality of explanations. In this case, neuron
activations due to a given sensor may cause neurons for other sensors to be
(de)activated. In effect, for a system where one sensor is drifting, this may
lead to the Centralised BAE explaining that other sensors are also drifting
when in fact, they are not.

To alleviate this, we propose a scheme of training an independent BAE model
for every sensor (Fig. 2(b)) called Coalitional BAE. By doing so, we force the
BAE outputs to be computed independently for each sensor (e.g attribution
score for $s_{1}$ is dependent only on input data from $s_{1}$ and exclude
other sensors).

One advantage of our proposed configuration is its simplicity of
implementation — no modification in model architecture is required when
converting from a centralised setup. During testing phase, we simply
concatenate the outputs of the independent BAEs which result in the same
dimensions as the Centralised BAE outputs. The concept of a Coalitional BAE is
inspired by coalitional agent-based system theory [33, 34] where each sensor
is represented by an independent BAE agent. In our context, the BAE agents
cooperate in a coalition to achieve a common goal of monitoring the
equipment’s health. We note that Lundberg and Lee [19] also leveraged
Shapley’s value from agent-based system theory to fairly allocate attribution
scores to the features.

Furthermore, our configuration is different from the multi-headed architecture
proposed by Canizo et al. [35] which does not guarantee the independence of
reconstructed signals since it splits only the early convolution layers while
the subsequent layers are fused with recurrent or fully-connected layers.

###  3.3 Evaluation Metrics for Sensor Attribution based on Covariate Shift

An important concept in evaluating the performance of explanation methods is
the covariate shift of sensor data in non-stationary environment [36].
Specifically, we consider covariate shift as the temporal departure of test
data from an initial training data distribution (for example, the healthy
phase of a manufacturing equipment). Therefore, we emphasise that a good
explanation outcome should be reflective of the underlying covariate shift
states of the sensors (i.e either shifting or non-shifting).

Two properties of a sensor attribution score are of relevance; its monotonic
relationship with $Y$, and its ability to rank sensor importance. Intuitively,
we expect a good explanation method to assign monotonically increasing
attribution scores to shifting sensors ($S^{shift}$) and conversely, the
attribution scores of non-shifting sensors ($S^{\bot{shift}}$) should be non-
monotonic. We also expect these scores to be higher in ranks for $S^{shift}$
when compared to those of $S^{\bot{shift}}$. Hence, we propose metrics to
evaluate these desired behaviours of good explanations.

####  3.3.1 Notation

We denote the set of $I$ shifting sensors as
$S^{shift}=\\{s^{shift}_{i}\\}_{i=1}^{I},S^{shift}\subset S_{K}$ and $J$ non-
shifting sensors as
$S^{\bot{shift}}=\\{s^{\bot{shift}}_{j}\\}_{j=1}^{J=K-I},S^{\bot{shift}}\subset
S_{K}$ and such that they are disjoint sets $S^{shift}\cap
S^{\bot{shift}}=\emptyset$.

Where the unobserved degradation of an equipment is of relevance,
$Y=[y_{n}]_{n=1}^{N},y_{n}\in[0,1]$ where $y_{n}=0$ represents perfect health
and $y_{n}$ increases linearly with $n$ until failure occurs at $y_{n}=1$. The
HI is a model of $Y$ and ideally has a monotonic relationship with $Y$.

Let $R(\textbf{a})\mathrel{\mathop{\mathchar 58\relax}}\rm
I\\!R^{dim(\textbf{a})}\to\\{r\mathrel{\mathop{\mathchar
58\relax}}r\in\mathbb{N}^{+},r\leq dim(\textbf{a})\\}^{dim(\textbf{a})}$ be
the ranking function which returns the ranks of elements in an arbitrary
vector. For example, $R([0.8,0.1,0.2])=[1,3,2]$. The relation between $i$-th
and $j$-th elements of the ranked vectors
$R(\textbf{a})_{i}<R(\textbf{a})_{j}$ indicates that $a_{i}$ has a higher rank
than $a_{j}$.

####  3.3.2 G-mean of Spearman drift coefficients, $G_{SDC}$

The Spearman rank correlation coefficient [37] measures the degree of any
monotonic relationship, either linear or non-linear, between two variables. As
a non-parametric technique, it does not assume normal distribution in the data
and is less sensitive to outliers compared to the Pearson correlation
coefficient [38]. Essentially, the Spearman rank correlation coefficient is
the Pearson correlation coefficient which operates on the ranks of variables
rather than the raw values. That is,
$Spearman(\textbf{a},\textbf{b})=Pearson(R(\textbf{a}),R(\textbf{b}))$ where
$Spearman$ is a function returning the Spearman rank correlation coefficient
between any vectors a and b, while the $Pearson$ function computes the Pearson
correlation coefficient. Next, the Spearman drift coefficient of the $k$-th
sensor attribution score can be defined as follows,

$\rho{({s_{k}})}=\left\\{\begin{array}[]{ll}\mathinner{\\!\left\lvert{Spearman}([\phi_{n,k}]_{n=1}^{N},Y)\right\rvert}&\quad{},\>\text{p\scalebox{0.75}[1.0]{$-$}{}value}\leq\alpha{}\\\
0&\quad{},\>\text{p\scalebox{0.75}[1.0]{$-$}{}value}>\alpha{}\end{array}\right.$
(8)

It takes on the absolute value of the Spearman rank correlation coefficient
when the p-value is less than or equal to a level of significance $\alpha$
(set as 0.05) and 0 otherwise which implies there is no monotonic
relationship. As a balanced evaluation metric, we compute the geometric mean
(G-mean) of the Spearman drift coefficients for $S^{shift}$ and
$S^{\bot{shift}}$,

$G_{SDC}=\sqrt{(\frac{1}{I}\sum_{i=1}^{I}{{\rho{(s^{shift}_{i})}}})(\frac{1}{J}\sum_{j=1}^{J}{1-{\rho{(s^{\bot{shift}}_{j})}})}},\>G_{SDC}\in[0,1]$
(9)

The first term in the square root favours explanation methods which have
higher Spearman drift coefficients on $S^{shift}$, while the second term
penalises methods which falsely assign high Spearman drift coefficients to
$S^{\bot{shift}}$.

####  3.3.3 G-mean of sensitivity-specificity of explanation ranking,
$G_{SSER}$

To evaluate the explanation methods in ranking the sensors importance, now we
turn to obtaining ranks of the sensor attribution scores across the set of
sensors. Using these set of ranks, we assign positive labels (shifting) to the
top-$I$ sensors, and negative labels (non-shifting) otherwise,

$top\scalebox{0.75}[1.0]{$-$}{I(r)}=\left\\{\begin{array}[]{ll}1&\quad{},\>r\leq
I\\\ 0&\quad{},\>r>I\end{array}\right.$ (10)

We obtain the assigned binary labels
$\textbf{A}=[\>top\scalebox{0.75}[1.0]{$-$}{I(r)}\mathrel{\mathop{\mathchar
58\relax}}r\in
R([\phi_{n,k}]_{k=1}^{K})\>]_{n=1}^{N},\textbf{A}\in\\{0,1\\}^{N\times{K}}$.
As one method to obtain gold labels, we define the function

$gold(k)=\left\\{\begin{array}[]{ll}1&\quad{},\>s_{k}\in S^{shift}\\\
0&\quad{},\>s_{k}\in S^{\bot{shift}}\end{array}\right.$ (11)

which leads to $\textbf{G}=[\>[\>gold(k)\>]_{k=1}^{K}\>]_{n=1}^{N}$,
$\textbf{G}\in\\{0,1\\}^{N\times{K}}$. To compare A and G, we vectorise them
and compute the confusion matrix. Subsequently, as a single performance
measure, we calculate the G-mean of the sensitivity-specificity of explanation
ranking,

$G_{SSER}=\sqrt{\underbrace{\frac{TP}{TP+FN}}_{sensitivity}\,\underbrace{\frac{TN}{TN+FP}}_{specificity}},\>G_{SSER}\in[0,1]$
(12)

where $TP$, $TN$, $FP$, and $FN$ are the true positives, true negatives, false
positives and false negatives respectively. We prefer the G-mean over the
$F_{1}$-measure since the $F_{1}$-measure does not account for $TN$ which is
relevant in evaluating the ranking of the non-shifting sensors [39]. Moreover,
the G-mean of sensitivity-specificity is a common evaluation metric for
imbalanced dataset classification [40]. This is relevant in our case, where
the number of sensors in $S^{shift}$ can potentially be higher (lower) than
$S^{\bot{shift}}$.

####  3.3.4 Sensor Explanation Quality Index , $SEQI$

Solely evaluating an explanation method using either $G_{SDC}$ or $G_{SSER}$
is not sufficient; it may perform poorly on either $G_{SDC}$ or $G_{SSER}$
which is not desirable (see Fig. 3(a), (b) and (c)) while the ideal quality is
to have high values on both metrics as depicted in Fig. 3(d).

Figure 3: Non-ideal explanation quality depicted in (a), (b) and (c). Low
$G_{SDC}$ is attained when the attribution score for $s^{\bot{shift}}$ has a
strong monotonic relationship with equipment degradation in (a) and (c), while
explanation methods with poor $G_{SSER}$ are shown in (a) and (b) where the
attribution score of $s^{\bot{shift}}$ is consistently higher than
$s^{shift}$. (d) depicts the desired explanation quality with high $G_{SDC}$
and $G_{SSER}$. Samples are obtained from evaluation on PRONOSTIA dataset.

Therefore, we propose to combine $G_{SDC}$ and $G_{SSER}$ as a weighted sum
for a more comprehensive evaluation metric which we call Sensor Explanation
Quality Index ($SEQI$),

$SEQI=\omega_{1}{G_{SDC}}+\omega_{2}{G_{SSER}},\>SEQI\in[0,1]$ (13)

In our study, we weigh both terms equally by setting $\omega_{1}$ and
$\omega_{2}$ as 0.5. Although our proposed metrics may not capture all
possible desiderata of explanations, they can still serve as useful sanity
check to compare explanation methods.

####  3.3.5 Obtaining labels for test set

One challenge to compute $G_{SDC}$, $G_{SSER}$ and $SEQI$ is in obtaining
temporal datasets with gold labels of which subset of sensors are shifting and
non-shifting. We describe a simple preprocessing procedure to augment these
labels from existing datasets.

Firstly, we designate a subset of sensors as $S^{shift}$ and the remaining as
$S^{\bot{shift}}$. Then, it is straightforward to obtain the measurements for
$S^{shift}$ by directly using the natural data which arise from a non-
stationary environment (e.g equipment run-to-failure). Meanwhile for the
$S^{\bot{shift}}$, we employ a bootstrap sampling from the training set to
maintain an equal number of data points as the original test set. This step
ensures data stationarity for the $S^{\bot{shift}}$. Using the resampled test
set and gold labels, we can proceed to computing the evaluation metrics
$G_{SDC}$, $G_{SSER}$ and $SEQI$. To have a more comprehensive evaluation, we
should repeat the evaluation procedures by designating a different combination
of shifting and $S^{\bot{shift}}$.

One important outcome of the resampling procedure is that it enforces the
$S^{\bot{shift}}$ raw inputs to be uncorrelated with the $S^{shift}$. This
does not enforce, however, the outputs of BAE to be uncorrelated.

##  4 Experimental setup

This section describes the experiment setup necessary to reproduce our results
of comparing various explanation methods and BAE configurations.

###  4.1 Dataset

We experimented with the proposed methods on two public datasets: PRONOSTIA
[41] and ZEMA [42, 43] which consist of sensor measurements collected from
multiple equipments’ run-to-failure. Each trajectory in these datasets is
prepared in an array with the following dimensions: ($N$ cycles $\times$ $K$
sensors $\times$ $D$ measurements).

For the PRONOSTIA dataset, the data were collected on a bearing testbed under
three different load conditions with multiple repetitions resulting in a total
of 17 trajectories. There were 2 accelerometers (1 horizontal and 1 vertical),
each of 25.6kHz sampling frequency. Every cycle had an interval of 10 seconds
and each sensor recorded 2560 measurements within a time window which lasted
1/10 of a second.

For the ZEMA dataset, 3 run-to-failure repetitions were conducted on a
electromechanical cylinder (EMC) testbed. A total of 11 sensors consisting of
a microphone, 3 vibration sensors mounted on the plain bearing, piston rod,
and ball bearing respectively, 1 axial force sensor, 1 pressure sensor, 1
velocity sensor, 1 active current sensor, and 3 motor current phase sensors
were installed. In the provided dataset, only the last 1 second of return
stroke is available and all measurements were resampled to 2kHz.

###  4.2 BAE Architecture

We implemented the BAE using the baetorch package[44] with the parameters and
layers stated in Table 1. The size of the ensemble was set to 5. For the
Centralised BAE, the input channel size of the first convolution layer is set
to the number of sensors present in each dataset. Similar architecture was
used for each BAE agent in the Coalitional BAE setup while fixing the input
channel size of the first convolution layer to 1.

Table 1: Encoder architecture of BAE for PRONOSTIA and ZEMA datasets. The decoder is a reflection of the encoder without the dense latent layer, and the convolution layers are replaced by convolution transpose layers. Layer | Output channels/nodes | Kernel size | Strides
---|---|---|---
1D-Convolution | 8 | 20 | 4
1D-Convolution | 5 | 10 | 2
1D-Convolution | 3 | 5 | 1
Flatten | - | - | -
Dense | 100 | - | -
Dense (latent) | 20 | - | -

The leaky ReLu [45] was used as the activation function for each layer with a
slope of 0.01. The sigmoid function was applied to scale each reconstructed
signal to [0,1].

###  4.3 Data pipeline

With the datasets and BAE architecture in Section 4.1 and Section 4.2, we
implemented the data pipeline using agentMET4FOF package [46] for managing the
data flow and model classes in a modular and reproducible way (Appendix A).

The data pipeline is executed repeatedly using various hyperparameters,
explanation methods and model configurations. We refer the model capacity as
the number of output channels and nodes of each layer, and the kernel size of
the convolution layers. The data pipeline has the BAE model capacity evaluated
at three levels: x$\frac{1}{2}$, x1 and x2 for both Centralised and
Coalitional BAEs. At each model capacity level, we vary the depth of the BAE
by setting the number of convolution layers at three levels: 1 layer, 2 layers
and 3 layers.

We also repeat the experiment runs by replacing the BAE with a deterministic
AE (size of ensemble as 1). For comparison against posthoc explanation
methods, we apply DeepLift [16] and GradientSHAP [19] on the deterministic AE
implemented using the Captum package***https://captum.ai to obtain sensor
attribution scores for the reconstruction loss. The data pipeline is
illustrated in Fig. 4 and described as follows:

Figure 4: Data pipeline for evaluating BAE explanation quality

  1. 1.

The raw multi-sensor data in the run-to-failure trajectory was preprocessed in
the form of array with dimensions of ($N$ cycles $\times$ $K$ sensors $\times$
$D$ measurements).

  2. 2.

We removed the first 10% and last 5% of the cycles which correspond to the
run-in and total failure phases respectively. This is because the measurements
during these phases can be highly transient and unstable. Moreover, it is of
less practical value for a HI which is too near the point of failure.

  3. 3.

We applied fast Fourier transform [47] to obtain the absolute amplitudes in
the frequency spectrum. This reduces the total feature size by half.

  4. 4.

Without shuffling the data, we split the trajectory and use the first 15-25%
of the cycles for training and remaining 25-95% for testing.

  5. 5.

During the training phase, we fitted a Min-Max scaler [48] on the training set
to rescale the feature values independently to [0,1]. Then, we trained the BAE
on the rescaled data using either the centralised or coalitional
configuration. We ran an automatic learning rate finder [49] and minimised the
loss in Eq. 2 using Adam optimiser [31] with $\lambda{}$ set to 0.001. The
training lasted for 250 epochs.

  6. 6.

For the test set, we firstly decided on a subset of sensors to be labelled as
shifting ($S^{shift}$) while the remaining sensors were labelled as non-
shifting ($S^{\bot{shift}}$). Data for $S^{\bot{shift}}$ was bootstrap sampled
from the training set to have equal number of data points as the test set
which ensures the data is always from the training distribution. The
$S^{shift}$ have their data unchanged from the test set.

  7. 7.

After preparing the test set, we fed the data to the Min-Max scaler and the
BAE. We obtained the sensor attribution scores using one of the explanation
methods.

  8. 8.

As a postprocessing step, simple moving average was applied on the sensor
attribution scores with a window size equal to the total cycles in the
training dataset. We found it helpful to reduce fluctuations from cycle-to-
cycle for a more stable result. For non-posthoc explanation methods such as
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ and
$\mathrm{Var_{\theta}(NLL)}$, the attribution scores were further normalised
to begin from 1 by dividing them by the mean values computed from the training
set.

  9. 9.

The $G_{SDC}$, $G_{SSER}$ , $SEQI$ are evaluated. To investigate the
correlation in explanations, we also calculated the Pearson correlation
coefficient between the sensor attribution scores of $S^{shift}$ and
$S^{\bot{shift}}$. For ZEMA, we took the sensor attribution scores averaged
across $S^{shift}$ and $S^{\bot{shift}}$.

  10. 10.

Steps 6 to 9 were repeated with varying combinations of $S^{shift}$ and
$S^{\bot{shift}}$. For PRONOSTIA, we repeated using all possible combinations
for 2 sensors. For ZEMA, due to the large number of all possible combinations
for 11 sensors, we evaluated using combinations of 1 and 10 shifting sensors
resulting in 22 combinations.

  11. 11.

Upon completing all evaluations on the combinations of $S^{shift}$ and
$S^{\bot{shift}}$, we repeat steps 1 to 10 with another run-to-failure
trajectory from the PRONOSTIA or ZEMA datasets.

##  5 Results and Discussion

We present our results of investigating the presence of correlation in the
sensor attribution scores. Then, we move on to a comprehensive comparison
between various BAE configurations and explanation methods based on the
proposed metrics of $G_{SDC}$, $G_{SSER}$ and $SEQI$. Additional results are
shown in Appendix B.

###  5.1 Correlation in BAE’s explanations

Surprisingly, from Fig. 5, we observe that the Centralised BAE induces strong
correlation between its explanations : the attribution scores for $S^{shift}$
take on similar shape as $S^{\bot{shift}}$ and the Pearson correlation
coefficient is high ($\geq{0.80}$) on samples from both PRONOSTIA and ZEMA.
This applies to both $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ and
$\mathrm{Var_{\theta}(NLL)}$ methods. This observation is crucial, it implies
these attribution scores can yield misleading explanations that some sensors
are monotonically shifting away from their healthy data distribution when in
fact, they are not. It was due to this observation that we proposed the
Coalitional BAE which enforces independence in the sensor attribution scores,
thereby reducing the correlation dramatically.

Another observation is that the absolute values of attribution scores for
$S^{\bot{shift}}$ are also increased for Centralised BAE due to the
correlation, and are higher than those of Coalitional BAE. As such, with the
smaller differences between attribution scores for $S^{shift}$ and
$S^{\bot{shift}}$, the ability to rank sensor importance may also be poorer
with the use of the centralised configuration.

(a) PRONOSTIA

(b) ZEMA

Figure 5: Samples from (a) PRONOSTIA and (b) ZEMA datasets showing the
attribution scores for $S^{shift}$ and $S^{\bot{shift}}$ obtained with the
Centralised and Coalitional BAEs using the
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ and
$\mathrm{Var_{\theta}(NLL)}$ explanation methods. Top and bottom rows in each
panel are different depictions of the same data. Color bar of bottom row
indicate the equipment degradation. The values of Pearson correlation
coefficients are also shown in the bottom row.

We further investigate this phenomenon with the complementary empirical
distribution function depicted in Fig. 6. It is confirmed that the centralised
configuration (solid lines) has consistently greater chances of exhibiting
high correlation between the attribution scores for $S^{shift}$ and
$S^{\bot{shift}}$ than the coalitional configuration (dotted lines).

Figure 6: Complementary empirical distribution function of the Pearson
correlation coefficient between sensor attribution scores of $S^{shift}$ and
$S^{\bot{shift}}$ for (a) PRONOSTIA and (b) ZEMA datasets for various BAE
configurations and explanation methods. Thick vertical line is drawn at r=0.8
which is deemed as high Pearson correlation coefficient threshold. The
empirical probability is evaluated over all experimental runs.

Strikingly, the Centralised BAE with $\mathrm{Var_{\theta}(NLL)}$ explanations
have more than half of the samples with high correlation on both PRONOSTIA and
ZEMA. In contrast, the Coalitional BAE counterpart has very close to zero
sample with high correlation. Furthermore, applying the posthoc explanation
methods DeepLift and GradientSHAP on the Centralised AE also exhibit high
chances of strong correlation in their attribution scores.

###  5.2 Quantitative evaluation of explanation quality

We tabulate the results of our experiments (Table 2, Table 3, Table 4) and
further analyse the differences between various BAE configurations and
explanation methods by visualising the critical difference diagrams (Fig. 7)
which are commonly used to compare machine learning models’ performances
across multiple datasets and hyperparameter settings [50, 51]. Essentially,
the critical difference diagram reports the results of a pairwise posthoc
analysis. For this, we adopt the non-parametric Wilcoxon signed-rank test [52]
after failing to reject the null hypothesis of a Friedman test [53].

Table 2: Mean and standard error of $G_{SDC}$ (0 = worst score, 1 = perfect
score) for various explanation methods with varied number of convolution
layers and model capacities evaluated on (a) PRONOSTIA and (b) ZEMA datasets.
Highest mean value of each column is bolded. * indicates overall highest
value.

(a) PRONOSTIA

|  |  |  G-mean of Spearman drift coefficients ($G_{SSC}$)
---|---|---|---
|  |  | 1 Convolution Layer | 2 Convolution Layers | 3 Convolution Layers
Method \ Capacity |  x$\frac{1}{2}$ |  x$1$ |  x$2$ |  x$\frac{1}{2}$ |  x$1$ |  x$2$ |  x$\frac{1}{2}$ |  x$1$ |  x$2$
AE | Central | $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ |  0.606$\pm{}$0.038 |  0.566$\pm{}$0.035 |  0.584$\pm{}$0.039 |  0.594$\pm{}$0.038 |  0.547$\pm{}$0.041 |  0.552$\pm{}$0.037 |  0.653$\pm{}$0.036 |  0.633$\pm{}$0.039 |  0.573$\pm{}$0.043
|  | DeepLIFT |  0.500$\pm{}$0.042 |  0.419$\pm{}$0.035 |  0.396$\pm{}$0.040 |  0.487$\pm{}$0.048 |  0.366$\pm{}$0.044 |  0.407$\pm{}$0.040 |  0.537$\pm{}$0.043 |  0.452$\pm{}$0.035 |  0.386$\pm{}$0.040
|  | GradShap |  0.558$\pm{}$0.030 |  0.522$\pm{}$0.034 |  0.531$\pm{}$0.030 |  0.529$\pm{}$0.031 |  0.511$\pm{}$0.023 |  0.481$\pm{}$0.030 |  0.530$\pm{}$0.026 |  0.489$\pm{}$0.028 |  0.487$\pm{}$0.032
| Coalition | $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ |  0.714$\pm{}$0.037 |  *0.742$\pm{}$0.031 |  0.715$\pm{}$0.028 |  0.668$\pm{}$0.034 |  0.694$\pm{}$0.031 |  0.698$\pm{}$0.033 |  0.682$\pm{}$0.032 |  0.661$\pm{}$0.043 |  0.707$\pm{}$0.030
BAE | Central | $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ |  0.527$\pm{}$0.037 |  0.527$\pm{}$0.032 |  0.557$\pm{}$0.037 |  0.574$\pm{}$0.037 |  0.560$\pm{}$0.041 |  0.525$\pm{}$0.036 |  0.676$\pm{}$0.035 |  0.598$\pm{}$0.041 |  0.603$\pm{}$0.038
|  | $\mathrm{Var_{\theta}(NLL)}$ |  0.456$\pm{}$0.037 |  0.449$\pm{}$0.033 |  0.467$\pm{}$0.038 |  0.370$\pm{}$0.040 |  0.425$\pm{}$0.045 |  0.336$\pm{}$0.044 |  0.701$\pm{}$0.040 |  0.404$\pm{}$0.041 |  0.358$\pm{}$0.044
| Coalition | $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ |  0.682$\pm{}$0.038 |  0.716$\pm{}$0.029 |  0.719$\pm{}$0.030 |  0.638$\pm{}$0.041 |  0.660$\pm{}$0.038 |  0.680$\pm{}$0.039 |  0.673$\pm{}$0.033 |  0.691$\pm{}$0.033 |  0.696$\pm{}$0.032
|  | $\mathrm{Var_{\theta}(NLL)}$ |  0.649$\pm{}$0.041 |  0.630$\pm{}$0.036 |  0.613$\pm{}$0.035 |  0.712$\pm{}$0.034 |  0.696$\pm{}$0.032 |  0.655$\pm{}$0.037 |  0.705$\pm{}$0.036 |  0.726$\pm{}$0.034 |  0.685$\pm{}$0.033

(b) ZEMA  G-mean of Spearman drift coefficients ($G_{SDC}$) 1 Convolution
Layer 2 Convolution Layers 3 Convolution Layers Method \ Capacity
x$\frac{1}{2}$ x$1$ x$2$ x$\frac{1}{2}$ x$1$ x$2$ x$\frac{1}{2}$ x$1$ x$2$ AE
Central $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.643$\pm{}$0.023
0.620$\pm{}$0.023 0.595$\pm{}$0.025 0.604$\pm{}$0.025 0.619$\pm{}$0.026
0.574$\pm{}$0.025 0.624$\pm{}$0.024 0.587$\pm{}$0.025 0.583$\pm{}$0.024
DeepLIFT 0.436$\pm{}$0.022 0.454$\pm{}$0.025 0.436$\pm{}$0.026
0.478$\pm{}$0.022 0.462$\pm{}$0.025 0.442$\pm{}$0.025 0.562$\pm{}$0.029
0.474$\pm{}$0.025 0.501$\pm{}$0.024 GradShap 0.490$\pm{}$0.020
0.458$\pm{}$0.021 0.454$\pm{}$0.023 0.540$\pm{}$0.022 0.570$\pm{}$0.022
0.486$\pm{}$0.025 0.578$\pm{}$0.026 0.526$\pm{}$0.022 0.567$\pm{}$0.024
Coalition $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$
0.648$\pm{}$0.024 0.659$\pm{}$0.025 0.672$\pm{}$0.024 0.659$\pm{}$0.021
0.647$\pm{}$0.025 0.640$\pm{}$0.024 0.654$\pm{}$0.021 0.646$\pm{}$0.022
0.627$\pm{}$0.027 BAE Central
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.638$\pm{}$0.023
0.604$\pm{}$0.025 0.578$\pm{}$0.026 0.615$\pm{}$0.024 0.597$\pm{}$0.025
0.548$\pm{}$0.026 0.644$\pm{}$0.024 0.592$\pm{}$0.024 0.600$\pm{}$0.024
$\mathrm{Var_{\theta}(NLL)}$ 0.560$\pm{}$0.022 0.498$\pm{}$0.020
0.473$\pm{}$0.019 0.533$\pm{}$0.017 0.505$\pm{}$0.021 0.493$\pm{}$0.021
0.566$\pm{}$0.018 0.516$\pm{}$0.015 0.528$\pm{}$0.020 Coalition
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.659$\pm{}$0.024
0.675$\pm{}$0.026 0.675$\pm{}$0.024 0.653$\pm{}$0.023 0.648$\pm{}$0.024
0.647$\pm{}$0.023 0.643$\pm{}$0.023 0.648$\pm{}$0.023 0.632$\pm{}$0.023
$\mathrm{Var_{\theta}(NLL)}$ *0.680$\pm{}$0.019 0.621$\pm{}$0.020
0.650$\pm{}$0.020 0.645$\pm{}$0.019 0.660$\pm{}$0.019 0.620$\pm{}$0.021
0.595$\pm{}$0.019 0.631$\pm{}$0.020 0.631$\pm{}$0.020

Table 3: Mean and standard error of $G_{SSER}$ (0 = worst score, 1 = perfect
score) for various explanation methods with varied number of convolution
layers and model capacities evaluated on (a) PRONOSTIA and (b) ZEMA datasets.
Highest mean value of each column is bolded. * indicates overall highest
value.

(a) PRONOSTIA  G-mean of sensitivity-specificity of explanation ranking
($G_{SSER}$) 1 Convolution Layer 2 Convolution Layers 3 Convolution Layers
Method \ Capacity x$\frac{1}{2}$ x$1$ x$2$ x$\frac{1}{2}$ x$1$ x$2$
x$\frac{1}{2}$ x$1$ x$2$ AE Central
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.769$\pm{}$0.053
0.826$\pm{}$0.051 0.933$\pm{}$0.031 0.621$\pm{}$0.058 0.692$\pm{}$0.057
0.681$\pm{}$0.056 0.589$\pm{}$0.059 0.629$\pm{}$0.053 0.628$\pm{}$0.056
DeepLIFT 0.496$\pm{}$0.064 0.585$\pm{}$0.063 0.651$\pm{}$0.065
0.480$\pm{}$0.064 0.475$\pm{}$0.063 0.495$\pm{}$0.066 0.490$\pm{}$0.066
0.494$\pm{}$0.064 0.497$\pm{}$0.065 GradShap 0.610$\pm{}$0.080
0.655$\pm{}$0.075 0.737$\pm{}$0.067 0.704$\pm{}$0.066 0.687$\pm{}$0.068
0.697$\pm{}$0.067 0.660$\pm{}$0.062 0.655$\pm{}$0.064 0.729$\pm{}$0.064
Coalition $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$
0.818$\pm{}$0.050 0.883$\pm{}$0.038 0.977$\pm{}$0.010 0.600$\pm{}$0.059
0.713$\pm{}$0.060 0.718$\pm{}$0.059 0.597$\pm{}$0.057 0.624$\pm{}$0.056
0.691$\pm{}$0.052 BAE Central
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.754$\pm{}$0.057
0.826$\pm{}$0.049 0.874$\pm{}$0.041 0.604$\pm{}$0.060 0.679$\pm{}$0.059
0.661$\pm{}$0.058 0.606$\pm{}$0.056 0.610$\pm{}$0.058 0.639$\pm{}$0.057
$\mathrm{Var_{\theta}(NLL)}$ 0.767$\pm{}$0.057 0.800$\pm{}$0.058
0.856$\pm{}$0.044 0.605$\pm{}$0.064 0.677$\pm{}$0.062 0.652$\pm{}$0.059
0.727$\pm{}$0.055 0.692$\pm{}$0.061 0.637$\pm{}$0.060 Coalition
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.808$\pm{}$0.053
0.917$\pm{}$0.035 *0.985$\pm{}$0.008 0.654$\pm{}$0.061 0.681$\pm{}$0.059
0.725$\pm{}$0.059 0.593$\pm{}$0.059 0.679$\pm{}$0.058 0.669$\pm{}$0.055
$\mathrm{Var_{\theta}(NLL)}$ 0.641$\pm{}$0.061 0.584$\pm{}$0.065
0.679$\pm{}$0.067 0.821$\pm{}$0.040 0.728$\pm{}$0.053 0.683$\pm{}$0.063
0.713$\pm{}$0.061 0.827$\pm{}$0.048 0.797$\pm{}$0.048

(b) ZEMA  G-mean of sensitivity-specificity of explanation ranking
($G_{SSER}$) 1 Convolution Layer 2 Convolution Layers 3 Convolution Layers
Method \ Capacity x$\frac{1}{2}$ x$1$ x$2$ x$\frac{1}{2}$ x$1$ x$2$
x$\frac{1}{2}$ x$1$ x$2$ AE Central
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.589$\pm{}$0.043
0.591$\pm{}$0.042 0.623$\pm{}$0.040 0.590$\pm{}$0.044 0.599$\pm{}$0.041
0.585$\pm{}$0.043 0.606$\pm{}$0.043 0.558$\pm{}$0.046 0.565$\pm{}$0.042
DeepLIFT 0.489$\pm{}$0.047 0.404$\pm{}$0.048 0.399$\pm{}$0.048
0.476$\pm{}$0.047 0.483$\pm{}$0.048 0.379$\pm{}$0.049 0.366$\pm{}$0.047
0.437$\pm{}$0.046 0.433$\pm{}$0.049 GradShap 0.151$\pm{}$0.041
0.142$\pm{}$0.040 0.157$\pm{}$0.041 0.173$\pm{}$0.042 0.176$\pm{}$0.042
0.173$\pm{}$0.042 0.217$\pm{}$0.048 0.169$\pm{}$0.042 0.203$\pm{}$0.047
Coalition $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$
0.677$\pm{}$0.035 0.719$\pm{}$0.031 0.688$\pm{}$0.034 0.647$\pm{}$0.038
0.679$\pm{}$0.034 0.654$\pm{}$0.037 0.635$\pm{}$0.041 0.672$\pm{}$0.035
0.622$\pm{}$0.041 BAE Central
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.590$\pm{}$0.043
0.613$\pm{}$0.041 0.624$\pm{}$0.040 0.598$\pm{}$0.040 0.586$\pm{}$0.042
0.589$\pm{}$0.042 0.541$\pm{}$0.044 0.581$\pm{}$0.042 0.579$\pm{}$0.043
$\mathrm{Var_{\theta}(NLL)}$ 0.542$\pm{}$0.044 0.566$\pm{}$0.043
0.580$\pm{}$0.043 0.522$\pm{}$0.050 0.539$\pm{}$0.044 0.542$\pm{}$0.048
0.491$\pm{}$0.048 0.572$\pm{}$0.044 0.566$\pm{}$0.044 Coalition
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.681$\pm{}$0.035
0.720$\pm{}$0.031 0.685$\pm{}$0.034 0.650$\pm{}$0.037 0.655$\pm{}$0.037
0.657$\pm{}$0.037 0.646$\pm{}$0.035 0.656$\pm{}$0.037 0.653$\pm{}$0.036
$\mathrm{Var_{\theta}(NLL)}$ 0.730$\pm{}$0.042 0.546$\pm{}$0.039
0.493$\pm{}$0.047 0.712$\pm{}$0.038 *0.799$\pm{}$0.024 0.542$\pm{}$0.044
0.531$\pm{}$0.046 0.627$\pm{}$0.048 0.577$\pm{}$0.039

Table 4: Mean and standard error of $SEQI$ (0 = worst score, 1 = perfect
score) for various explanation methods with varied number of convolution
layers and model capacities evaluated on (a) PRONOSTIA and (b) ZEMA datasets.
Highest mean value of each column is bolded. * indicates overall highest
value.

(a) PRONOSTIA  Sensor Explanation Quality Index ($SEQI$) 1 Convolution Layer 2
Convolution Layers 3 Convolution Layers Method \ Capacity x$\frac{1}{2}$ x$1$
x$2$ x$\frac{1}{2}$ x$1$ x$2$ x$\frac{1}{2}$ x$1$ x$2$ AE Central
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.687$\pm{}$0.031
0.696$\pm{}$0.032 0.759$\pm{}$0.025 0.608$\pm{}$0.044 0.619$\pm{}$0.043
0.616$\pm{}$0.036 0.621$\pm{}$0.041 0.631$\pm{}$0.039 0.600$\pm{}$0.043
DeepLIFT 0.498$\pm{}$0.038 0.502$\pm{}$0.034 0.524$\pm{}$0.034
0.484$\pm{}$0.043 0.420$\pm{}$0.041 0.451$\pm{}$0.035 0.514$\pm{}$0.040
0.473$\pm{}$0.034 0.442$\pm{}$0.038 GradShap 0.584$\pm{}$0.041
0.589$\pm{}$0.042 0.634$\pm{}$0.037 0.617$\pm{}$0.038 0.599$\pm{}$0.036
0.589$\pm{}$0.036 0.595$\pm{}$0.034 0.572$\pm{}$0.037 0.608$\pm{}$0.040
Coalition $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$
0.766$\pm{}$0.034 0.813$\pm{}$0.025 0.846$\pm{}$0.016 0.634$\pm{}$0.041
0.703$\pm{}$0.039 0.708$\pm{}$0.041 0.639$\pm{}$0.038 0.643$\pm{}$0.044
0.699$\pm{}$0.031 BAE Central
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.641$\pm{}$0.037
0.676$\pm{}$0.030 0.715$\pm{}$0.028 0.589$\pm{}$0.042 0.620$\pm{}$0.044
0.593$\pm{}$0.039 0.641$\pm{}$0.039 0.604$\pm{}$0.043 0.621$\pm{}$0.039
$\mathrm{Var_{\theta}(NLL)}$ 0.612$\pm{}$0.037 0.624$\pm{}$0.034
0.662$\pm{}$0.029 0.488$\pm{}$0.045 0.551$\pm{}$0.044 0.494$\pm{}$0.039
0.714$\pm{}$0.040 0.548$\pm{}$0.039 0.497$\pm{}$0.043 Coalition
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.745$\pm{}$0.038
0.816$\pm{}$0.025 *0.852$\pm{}$0.016 0.646$\pm{}$0.045 0.671$\pm{}$0.043
0.702$\pm{}$0.043 0.633$\pm{}$0.040 0.685$\pm{}$0.039 0.682$\pm{}$0.035
$\mathrm{Var_{\theta}(NLL)}$ 0.645$\pm{}$0.044 0.607$\pm{}$0.043
0.646$\pm{}$0.041 0.767$\pm{}$0.027 0.712$\pm{}$0.035 0.669$\pm{}$0.040
0.709$\pm{}$0.044 0.776$\pm{}$0.035 0.741$\pm{}$0.033

(b) ZEMA  Sensor Explanation Quality Index ($SEQI$) 1 Convolution Layer 2
Convolution Layers 3 Convolution Layers Method \ Capacity x$\frac{1}{2}$ x$1$
x$2$ x$\frac{1}{2}$ x$1$ x$2$ x$\frac{1}{2}$ x$1$ x$2$ AE Central
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.616$\pm{}$0.027
0.605$\pm{}$0.027 0.609$\pm{}$0.025 0.597$\pm{}$0.028 0.609$\pm{}$0.026
0.580$\pm{}$0.028 0.615$\pm{}$0.028 0.572$\pm{}$0.030 0.574$\pm{}$0.030
DeepLIFT 0.463$\pm{}$0.028 0.429$\pm{}$0.029 0.417$\pm{}$0.030
0.477$\pm{}$0.026 0.472$\pm{}$0.030 0.411$\pm{}$0.031 0.464$\pm{}$0.025
0.455$\pm{}$0.029 0.467$\pm{}$0.030 GradShap 0.321$\pm{}$0.025
0.300$\pm{}$0.023 0.306$\pm{}$0.023 0.357$\pm{}$0.026 0.373$\pm{}$0.024
0.329$\pm{}$0.024 0.398$\pm{}$0.031 0.348$\pm{}$0.026 0.385$\pm{}$0.029
Coalition $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$
0.663$\pm{}$0.026 0.689$\pm{}$0.025 0.680$\pm{}$0.025 0.653$\pm{}$0.026
0.663$\pm{}$0.026 0.647$\pm{}$0.027 0.644$\pm{}$0.027 0.659$\pm{}$0.025
0.625$\pm{}$0.029 BAE Central
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.614$\pm{}$0.027
0.608$\pm{}$0.026 0.601$\pm{}$0.026 0.606$\pm{}$0.027 0.592$\pm{}$0.027
0.568$\pm{}$0.028 0.593$\pm{}$0.029 0.586$\pm{}$0.029 0.590$\pm{}$0.029
$\mathrm{Var_{\theta}(NLL)}$ 0.551$\pm{}$0.025 0.532$\pm{}$0.024
0.527$\pm{}$0.024 0.528$\pm{}$0.029 0.522$\pm{}$0.025 0.517$\pm{}$0.030
0.529$\pm{}$0.028 0.544$\pm{}$0.025 0.547$\pm{}$0.026 Coalition
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.670$\pm{}$0.026
0.698$\pm{}$0.025 0.680$\pm{}$0.026 0.652$\pm{}$0.026 0.652$\pm{}$0.027
0.652$\pm{}$0.026 0.644$\pm{}$0.025 0.652$\pm{}$0.026 0.643$\pm{}$0.026
$\mathrm{Var_{\theta}(NLL)}$ 0.705$\pm{}$0.025 0.583$\pm{}$0.026
0.572$\pm{}$0.029 0.678$\pm{}$0.025 *0.729$\pm{}$0.019 0.581$\pm{}$0.027
0.563$\pm{}$0.029 0.629$\pm{}$0.029 0.604$\pm{}$0.026

(a) $G_{SDC}$

(b) $G_{SSER}$

(c) $SEQI$

Figure 7: Critical difference diagrams for comparing explanation methods under
evaluation metrics of (a) $G_{SDC}$, (b) $G_{SSER}$ and (c) $SEQI$. Average
rank of each method is shown next to the labels. The methods on the right have
higher ranks than the methods on left. A thick line connecting the methods
indicate there is no significant difference between the methods.

Coalitional versus centralised configuration. Both Coalitional BAE and AE with
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ methods are consistently
better than their centralised counterparts based on $G_{SDC}$ , $G_{SSER}$ and
$SEQI$ metrics as evident by the significant differences shown in Fig. 7. This
is further evidenced by the overall best score achievements by the Coalitional
BAE and AE on both PRONOSTIA and ZEMA datasets for all metrics in Table 2,
Table 3 and Table 4. Clearly, our proposed solution improved the explanation
quality by reducing correlation and enforcing independence in the
explanations.

Another implication of the Coalitional BAE is its highly parallelisable
computations. As smart sensor technology advances, every sensor can be
installed with a BAE software agent and cooperate with other agents in the
heterogeneous sensor network. This further facilitates the incorporation of
BAE into a flexible multi-agent system for industrial applications where new
sensors can be introduced or replaced any time [54].

$\boldsymbol{\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}}$ versus
$\boldsymbol{\mathrm{Var_{\theta}(NLL)}}$ as explanation methods. We do not
see a consistent difference between
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ and the
$\mathrm{Var_{\theta}(NLL)}$ methods. Although Fig. 7 indicate that both
Centralised and Coalitional BAEs with
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ yield significant
advantages over their counterparts with $\mathrm{Var_{\theta}(NLL)}$, when we
compare only the best performances in Table 2, Table 3 and Table 4, we note
that while the $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ is better
on PRONOSTIA, the $\mathrm{Var_{\theta}(NLL)}$ method yield better scores on
ZEMA with the Coalitional BAE.

One reason for this inconsistency could be that the explanation quality
depends on the dataset complexity. For instance, we observe the highest
$G_{SDC}$, $G_{SSER}$ and $SEQI$ achieved on ZEMA are much lower than those of
PRONOSTIA. Since there are more sensors in the ZEMA testbed (11 sensors), it
poses a more difficult problem to obtain higher quality explanations as
compared to PRONOSTIA testbed (2 sensors). Nonetheless, this presents a new
avenue for future research by combining multiple explanations methods and also
further understanding the conditions when one explanation method is better
than the other.

BAE versus AE. We posit that using the BAE is only clearly advantageous over
the AE when we use it with the coalitional configuration. In particular,
although results in Fig. 7 indicate that switching from a Centralised BAE to a
Centralised AE may actually harm the explanation quality, this pattern is
reversed when we compare Coalitional BAE against the Coalitional AE with
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$. Despite their differences
being not significant, the Coalitional BAE consistently yield better ranks
than Coalitional AE with $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$
on all evaluation metrics. Moreover, the $\mathrm{Var_{\theta}(NLL)}$ method
is only possible with the BAE in which it fared the best $G_{SDC}$ and $SEQI$
scores on ZEMA when used with the coalitional configuration (Table 2(b) and
Table 4(c)).

One criticism of the BAE is that it increases the number of computations by
$M$ size of the ensemble, as compared to the AE. While this is true, as
embedded systems continue advancing with dedicated architectures for neural
computing [55], the BAE can take advantage over these new technologies given
that the BAE can be highly parallelised since each model in the ensemble
trains and predicts independently. Furthermore, the highly efficient
randomised MAP sampling method further improves the practicality of obtaining
explanations with BAE.

Natural versus posthoc explanation methods. As proposed in Section 3.1.4, the
BAE is naturally equipped with explanation methods with the
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ and
$\mathrm{Var_{\theta}(NLL)}$ under the independent likelihood assumption.
Hence, the use of posthoc explanation method may not be necessary. From Fig.
7, DeepLift and GradientSHAP have significantly worst performances, with the
exception of $G_{SDC}$ where Centralised BAE with $\mathrm{Var_{\theta}(NLL)}$
is worse than the Centralised AE with GradientSHAP, but not with DeepLift.
Although the difference could be due to the centralised versus coalitional
configurations, to be able to compare this, we do not find a valid approach of
applying the posthoc explanation methods on the Coalitional BAE and AE. The
posthoc explanation methods are also much more computationally demanding as
compared to simple sum of arrays with the
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ and
$\mathrm{Var_{\theta}(NLL)}$ methods which deem them less practical for real-
time IIoT applications.

##  6 Conclusion

In this work, we presented a study on explainable HI with BAE which is a type
of unsupervised DNN formulated from a probabilistic perspective. In short, our
contributions can be summarised as follows: (1) proposal of two sensor
attribution methods natural to the BAE based on decomposition of
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ and
$\mathrm{Var_{\theta}(NLL)}$, (2) the development of Coalitional BAE, (3)
three quantitative metrics for evaluating quality of explanations and (4)
comprehensive experiments which compared the quality of explanations using
various BAE configurations and explanation methods.

The primary implication of our study is that we gained better understanding on
the explanation methods for BAE. We found that the BAE is naturally equipped
with explanation methods through its probabilistic formulation. This means it
is unnecessary to use posthoc explanation methods such as DeepLift and
GradientSHAP which we found to be empirically underperforming compared to our
proposed explanation methods.

With the quantitative evaluation metrics that this paper has established for
explainable deep learning, future studies can facilitate more rigorous
comparisons between explanation methods. Specifically, the proposed metrics
are (1) $G_{SDC}$ to capture the monotonic relationship between sensor
attribution scores and equipment degradation, (2) $G_{SSER}$ to evaluate the
ability to rank sensor importance and (3) $SEQI$ as a comprehensive measure of
HI explanation quality by encompassing both $G_{SDC}$ and $G_{SSER}$ through a
weighted sum.

Through our experiments, we found that the conventional training scheme of the
Centralised BAE yielded misleading explanations due to correlated outputs. For
instance, we observed that non-shifting sensors were assigned monotonically
increasing attribution scores and had similar shape as those of shifting
sensors. This is surprising, and we caution users against being overreliant on
the explanations arising from the Centralised BAEs without understanding when
sensors may be failing.

Motivated by this observation, we presented a configuration inspired by
coalitional agent-based system theory called Coalitional BAE to improve the
explanations of BAE’s predictions. The Coalitional BAE entails training an
independent BAE agent for every sensor mounted on the IIoT equipment, as
opposed to training a single BAE agent for all sensor data streams. The
advantage of Coalitional BAE is evident through the experiment results as we
consistently find the Coalitional BAE to yield the best scores on PRONOSTIA
and ZEMA datasets.

## Acknowledgment

The work reported here was supported by European Metrology Programme for
Innovation and Research (EMPIR) under the project Metrology for the Factory of
the Future (MET4FOF), project number 17IND12 and part-sponsored by Research
England’s Connecting Capability Fund award CCF18-7157 : Promoting the Internet
of Things via Collaboration between HEIs and Industry (Pitch-In). We express
our gratitude to Sascha Eichstädt for his inputs.

## References

  * [1] A. Ghorbani, A. Abid, J. Zou, Interpretation of neural networks is fragile, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, 2019, pp. 3681–3688.
  * [2] J. Wang, Y. Ma, L. Zhang, R. X. Gao, D. Wu, Deep learning for smart manufacturing: Methods and applications, Journal of Manufacturing Systems 48 (2018) 144–156, special Issue on Smart Manufacturing.  doi:https://doi.org/10.1016/j.jmsy.2018.01.003.
  * [3] B. X. Yong, Y. Fathy, A. Brintrup, Bayesian autoencoders for drift detection in industrial environments, in: 2020 IEEE International Workshop on Metrology for Industry 4.0 & IoT, IEEE, 2020, pp. 627–631.
  * [4] B. X. Yong, T. Pearce, A. Brintrup, Bayesian autoencoders: Analysing and fixing the bernoulli likelihood for out-of-distribution detection, in: International Conference on Machine Learning Workshop on Uncertainty and Robustness in Deep Learning, 2020.
  * [5] L. Ren, Y. Sun, J. Cui, L. Zhang, Bearing remaining useful life prediction based on deep autoencoder and deep neural networks, Journal of Manufacturing Systems 48 (2018) 71–77.
  * [6] M. Sohaib, J.-M. Kim, Reliable fault diagnosis of rotary machine bearings using a stacked sparse autoencoder-based deep neural network, Shock and Vibration 2018\.
  * [7] J. Ma, H. Su, W.-l. Zhao, B. Liu, Predicting the remaining useful life of an aircraft engine using a stacked sparse autoencoder with multilayer self-learning, Complexity 2018 (2018).
  * [8] Z. Meng, X. Zhan, J. Li, Z. Pan, An enhancement denoising autoencoder for rolling bearing fault diagnosis, Measurement 130 (2018) 448–454.
  * [9] P. Lin, J. Tao, A novel bearing health indicator construction method based on ensemble stacked autoencoder, in: 2019 IEEE International Conference on Prognostics and Health Management (ICPHM), IEEE, 2019, pp. 1–9.
  * [10] F. Xu, W. tai Peter Tse, Y. L. Tse, Roller bearing fault diagnosis using stacked denoising autoencoder in deep learning and gath–geva clustering algorithm without principal component analysis and data label, Applied Soft Computing 73 (2018) 898–913.  doi:https://doi.org/10.1016/j.asoc.2018.09.037.
  * [11] A. Anaissi, S. M. Zandavi, Multi-objective autoencoder for fault detection and diagnosis in higher-order data, in: 2019 International Joint Conference on Neural Networks (IJCNN), IEEE, 2019, pp. 1–8.
  * [12] D. Y. Oh, I. D. Yun, Residual error based anomaly detection using auto-encoder in smd machine sound, Sensors 18 (5) (2018) 1308.
  * [13] T. Amarbayasgalan, V. H. Pham, N. Theera-Umpon, K. H. Ryu, et al., Unsupervised anomaly detection approach for time-series in multi-domains using deep reconstruction error, Symmetry 12 (8) (2020) 1251.
  * [14] S. Zhang, S. Zhang, B. Wang, T. G. Habetler, Deep learning algorithms for bearing fault diagnosticsx—a comprehensive review, IEEE Access 8 (2020) 29857–29881.
  * [15] M. T. Ribeiro, S. Singh, C. Guestrin, ”why should i trust you?” explaining the predictions of any classifier, in: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 2016, pp. 1135–1144.
  * [16] A. Shrikumar, P. Greenside, A. Kundaje, Learning important features through propagating activation differences, in: International Conference on Machine Learning, 2017, pp. 3145–3153.
  * [17] M. Sundararajan, A. Taly, Q. Yan, Axiomatic attribution for deep networks, in: International Conference on Machine Learning, 2017, pp. 3319–3328.
  * [18] L. Antwarg, B. Shapira, L. Rokach, Explaining anomalies detected by autoencoders using shap, arXiv preprint arXiv:1903.02407 (2019).
  * [19] S. M. Lundberg, S.-I. Lee, A unified approach to interpreting model predictions, in: Advances in Neural Information Processing Systems, 2017, pp. 4765–4774.
  * [20] P. Bergmann, M. Fauser, D. Sattlegger, C. Steger, Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 9592–9600.
  * [21] M. Martinez-Garcia, Y. Zhang, J. Wan, J. Mcginty, Visually interpretable profile extraction with an autoencoder for health monitoring of industrial systems, in: 2019 IEEE 4th International Conference on Advanced Robotics and Mechatronics (ICARM), IEEE, 2019, pp. 649–654.
  * [22] P. Liznerski, L. Ruff, R. A. Vandermeulen, B. J. Franks, M. Kloft, K.-R. Müller, Explainable deep one-class classification, arXiv preprint arXiv:2007.01760 (2020).
  * [23] C.-K. Yeh, C.-Y. Hsieh, A. Suggala, D. I. Inouye, P. K. Ravikumar, On the (in) fidelity and sensitivity of explanations, in: Advances in Neural Information Processing Systems, 2019, pp. 10967–10978.
  * [24] S. Hooker, D. Erhan, P.-J. Kindermans, B. Kim, A benchmark for interpretability methods in deep neural networks, in: Advances in Neural Information Processing Systems, 2019, pp. 9737–9748.
  * [25] I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT Press, 2016.
  * [26] T. Pearce, F. Leibfried, A. Brintrup, Uncertainty in neural networks: Approximately bayesian ensembling, in: International conference on artificial intelligence and statistics, PMLR, 2020, pp. 234–244.
  * [27] C. Blundell, J. Cornebise, K. Kavukcuoglu, D. Wierstra, Weight uncertainty in neural network, in: International Conference on Machine Learning, PMLR, 2015, pp. 1613–1622.
  * [28] B. Lakshminarayanan, A. Pritzel, C. Blundell, Simple and scalable predictive uncertainty estimation using deep ensembles, in: Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017, pp. 6405–6416.
  * [29] J. Yao, W. Pan, S. Ghosh, F. Doshi-Velez, Quality of uncertainty quantification for bayesian neural network inference, arXiv preprint arXiv:1906.09686 (2019).
  * [30] K. He, X. Zhang, S. Ren, J. Sun, Delving deep into rectifiers: Surpassing human-level performance on imagenet classification, in: Proceedings of the IEEE international conference on computer vision, 2015, pp. 1026–1034.
  * [31] D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014).
  * [32] G. Chandrashekar, F. Sahin, A survey on feature selection methods, Computers & Electrical Engineering 40 (1) (2014) 16–28, 40th-year commemorative issue.  doi:https://doi.org/10.1016/j.compeleceng.2013.11.024.
  * [33] T. Rahwan, N. R. Jennings, An algorithm for distributing coalitional value calculations among cooperating agents, Artificial Intelligence 171 (8) (2007) 535–567.  doi:https://doi.org/10.1016/j.artint.2007.03.002.
  * [34] T. Rahwan, N. R. Jennings, Distributing coalitional value calculations among cooperating agents (2005).
  * [35] M. Canizo, I. Triguero, A. Conde, E. Onieva, Multi-head cnn–rnn for multi-time series anomaly detection: An industrial case study, Neurocomputing 363 (2019) 246–260.
  * [36] H. Raza, G. Prasad, Y. Li, Adaptive learning with covariate shift-detection for non-stationary environments, in: 2014 14th UK Workshop on Computational Intelligence (UKCI), IEEE, 2014, pp. 1–8.
  * [37] C. Spearman, The proof and measurement of association between two things, The American journal of psychology 100 (3/4) (1987) 441–471.
  * [38] T. D. Gauthier, Detecting trends using spearman’s rank correlation coefficient, Environmental forensics 2 (4) (2001) 359–362.
  * [39] D. Chicco, G. Jurman, The advantages of the matthews correlation coefficient (mcc) over f1 score and accuracy in binary classification evaluation, BMC genomics 21 (1) (2020) 1–13.
  * [40] L. I. Kuncheva, Á. Arnaiz-González, J.-F. Díez-Pastor, I. A. Gunn, Instance selection improves geometric mean accuracy: a study on imbalanced data classification, Progress in Artificial Intelligence 8 (2) (2019) 215–228.
  * [41] P. Nectoux, R. Gouriveau, K. Medjaher, E. Ramasso, B. Chebel-Morello, N. Zerhouni, C. Varnier, Pronostia: An experimental platform for bearings accelerated degradation tests., in: IEEE International Conference on Prognostics and Health Management, 2012.
  * [42] T. Dorst, Sensor data set of 3 electromechanical cylinder at ZeMA testbed (2kHz) (May 2019).  doi:10.5281/zenodo.3364431.
  * [43] N. Helwig, P. Merten, T. Schneider, A. Schütze, Integrated sensor system for condition monitoring of electromechanical cylinders, in: Multidisciplinary Digital Publishing Institute Proceedings, Vol. 1, 2017, p. 626\.
  * [44] B. X. Yong, baetorch : A python package for bayesian autoencoders, 2020, https://doi.org/10.5281/zenodo.4555443.  doi:10.5281/zenodo.4555443.
  * [45] A. L. Maas, A. Y. Hannun, A. Y. Ng, Rectifier nonlinearities improve neural network acoustic models, Citeseer.
  * [46] B. X. Yong, B. Ludwig, A. P. Vedurmudi, M. Gruber, H. Lulic, agentmet4fof : A python package for developing multi-agent system for iiot with uncertainty quantification, 2020, https://pypi.org/project/agentMET4FOF.
  * [47] E. O. Brigham, R. Morrow, The fast fourier transform, IEEE spectrum 4 (12) (1967) 63–70.
  * [48] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, E. Duchesnay, Scikit-learn: Machine learning in Python, Journal of Machine Learning Research 12 (2011) 2825–2830.
  * [49] L. N. Smith, Cyclical learning rates for training neural networks, in: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE, 2017, pp. 464–472.
  * [50] J. Demšar, Statistical comparisons of classifiers over multiple data sets, The Journal of Machine Learning Research 7 (2006) 1–30.
  * [51] H. Ismail Fawaz, G. Forestier, J. Weber, L. Idoumghar, P.-A. Muller, Deep learning for time series classification: a review, Data Mining and Knowledge Discovery 33 (4) (2019) 917–963.
  * [52] F. Wilcoxon, Individual comparisons by ranking methods, in: Breakthroughs in statistics, Springer, 1992, pp. 196–202.
  * [53] M. Friedman, A comparison of alternative tests of significance for the problem of m rankings, The Annals of Mathematical Statistics 11 (1) (1940) 86–92.
  * [54] B. X. Yong, A. Brintrup, Multi agent system for machine learning under uncertainty in cyber physical manufacturing system, in: International Workshop on Service Orientation in Holonic and Multi-Agent Manufacturing, Springer, 2019, pp. 244–257.
  * [55] T. Gong, T. Fan, J. Guo, Z. Cai, Gpu-based parallel optimization of immune convolutional neural network and embedded system, Engineering Applications of Artificial Intelligence 62 (2017) 384–395.  doi:https://doi.org/10.1016/j.engappai.2016.08.019.

##  Appendix A Data pipeline using agentMET4FOF

Implementation of the data pipeline using agentMET4FOF package is visualised
and the agent roles are described.

(a) Centralised BAE

(b) Coalitional BAE

Figure 8: agentMET4FOF network setup for (a) Centralised BAE and (b) Coalitional BAE. Each node depicts an agent and the edges denote the channels. Table 5: Agent classes, description of their roles Agents |  Roles
---|---
Datastream (e.g ZEMA_7) |  Loads one of the run-to-failure repetition from ZEMA and PRONOSTIA data sets in an array. The first 15% of the trajectory data set and last 5% were removed and splits into training and test sets. The test set contains the full data set in which the training portion will be removed by the Truncate Agent later for a fair evaluation.
Move Axis |  Swaps the array’s axes. Required to reshape the arrays from ( Cycles x Measurements x Sensors) to ( Cycles x Sensors x Measurements).
FFT |  Applies fast Fourier transform on the Measurements dimension when given an array of (Cycles x Sensors x Measurements).
MinMax Scaler |  Fits and transform the Min-Max scaler on the training data and also transforms the test data.
SetDrift |  With the specified sets of shifting sensors and non-shifting sensors, this agent resamples the data of the non-shifting sensors from the training set. While the shifting sensors’ data are left unchanged. This agent receives only the test set.
SelectSensor |  Used only for Coalitional BAE configuration. Selects one of the sensor indexed from the (Cycles x Sensors x Measurements) array such that an array of (Cycles x 1 x Measurements) is sent out.
BAE |  Fits the BAE on the training data. Outputs the negative log-likelihood samples in a 4D array of (BAE Samples x Cycles x Sensors x Measurements). If posthoc explanation method is enabled such as GradShap or DeepLift, the outputs are the attribution scores from this method.
Aggregator |  Used only for Coalitional BAE configuration. Aggregates and concatenates the data received from each BAE agent such that (BAE Samples x Cycles x Sensors x Measurements) array is recovered.
MeanVar |  Computes the mean and variance of the BAE samples and subsequently sums over the Measurements dimension of the array. Outputs the dictionary keyed by ”mean” and ”var”, each containing array of (Cycles x Sensors) shape.
SMA |  For each Sensor, applies simple moving average to smoothen the attribution scores across the Cycles dimension.
Normalise |  Normalises the sensor attribution scores by the mean of the training data.
Truncate |  Truncates the training data from the test set for evaluation.
Plotting |  Plots the samples as shown in Fig. 1.
PlottingCorr |  Plots the correlation between shifting and non-shifting sensors as shown in Fig. 5.
Spearman |  Calculates the $G_{SDC}$.
Ranking |  Calculates the $G_{SSER}$.
Monitor |  Renders the received data on dashboard.
Orchestrator |  Omniscient agent which controls the experiment repetitions. Varies the experimental parameters, sets of shifting and non-shifting sensors, and the BAE configurations.

##  Appendix B Additional results

Results on (1) absolute Pearson correlation coefficient between attribution
scores of $S^{shift}$ and $S^{\bot{shift}}$ and (2) absolute Matthew
correlation coefficient for sensor ranking (instead of $G_{SSER}$) are
tabulated and compared using critical difference diagrams.

Table 6: Mean and standard error of absolute Pearson correlation coefficient
between attribution scores of $S^{shift}$ and $S^{\bot{shift}}$ for various
explanation methods with varied number of convolution layers and model
capacities evaluated on (a) PRONOSTIA and (b) ZEMA datasets. Highest mean
value of each column is bolded. * indicates overall highest value.

(a) PRONOSTIA

|  |  | Pearson correlation coefficient
---|---|---|---
|  |  | 1 Convolution Layer | 2 Convolution Layers | 3 Convolution Layers
Method \ Capacity |  x$\frac{1}{2}$ |  x$1$ |  x$2$ |  x$\frac{1}{2}$ |  x$1$ |  x$2$ |  x$\frac{1}{2}$ |  x$1$ |  x$2$
AE | Central | $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ |  0.537$\pm{}$0.054 |  0.630$\pm{}$0.051 |  0.634$\pm{}$0.052 |  0.418$\pm{}$0.042 |  0.538$\pm{}$0.049 |  0.550$\pm{}$0.052 |  0.314$\pm{}$0.038 |  0.422$\pm{}$0.052 |  0.463$\pm{}$0.057
|  | DeepLIFT |  0.501$\pm{}$0.053 |  0.685$\pm{}$0.054 |  0.644$\pm{}$0.056 |  0.564$\pm{}$0.057 |  0.615$\pm{}$0.057 |  0.607$\pm{}$0.059 |  0.372$\pm{}$0.040 |  0.647$\pm{}$0.054 |  0.707$\pm{}$0.053
|  | GradShap |  0.667$\pm{}$0.035 |  0.605$\pm{}$0.056 |  0.545$\pm{}$0.063 |  0.575$\pm{}$0.052 |  0.722$\pm{}$0.041 |  0.755$\pm{}$0.040 |  0.573$\pm{}$0.048 |  0.694$\pm{}$0.047 |  0.691$\pm{}$0.052
| Coalition | $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ |  0.261$\pm{}$0.037 |  0.242$\pm{}$0.036 |  0.289$\pm{}$0.032 |  0.298$\pm{}$0.036 |  0.302$\pm{}$0.030 |  0.318$\pm{}$0.033 |  0.290$\pm{}$0.034 |  0.272$\pm{}$0.032 |  0.312$\pm{}$0.033
BAE | Central | $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ |  0.622$\pm{}$0.046 |  0.675$\pm{}$0.049 |  0.697$\pm{}$0.046 |  0.464$\pm{}$0.051 |  0.550$\pm{}$0.053 |  0.566$\pm{}$0.057 |  0.286$\pm{}$0.036 |  0.507$\pm{}$0.054 |  0.494$\pm{}$0.059
|  | $\mathrm{Var_{\theta}(NLL)}$ |  0.745$\pm{}$0.057 |  0.749$\pm{}$0.051 |  0.695$\pm{}$0.051 |  0.718$\pm{}$0.052 |  0.771$\pm{}$0.053 |  *0.846$\pm{}$0.037 |  0.388$\pm{}$0.046 |  0.800$\pm{}$0.045 |  0.773$\pm{}$0.048
| Coalition | $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ |  0.277$\pm{}$0.036 |  0.295$\pm{}$0.029 |  0.245$\pm{}$0.030 |  0.312$\pm{}$0.034 |  0.287$\pm{}$0.033 |  0.300$\pm{}$0.034 |  0.288$\pm{}$0.032 |  0.290$\pm{}$0.032 |  0.305$\pm{}$0.033
|  | $\mathrm{Var_{\theta}(NLL)}$ |  0.312$\pm{}$0.037 |  0.268$\pm{}$0.033 |  0.301$\pm{}$0.031 |  0.304$\pm{}$0.036 |  0.316$\pm{}$0.034 |  0.324$\pm{}$0.036 |  0.270$\pm{}$0.032 |  0.300$\pm{}$0.032 |  0.298$\pm{}$0.036

(b) ZEMA  Pearson correlation coefficient 1 Convolution Layer 2 Convolution
Layers 3 Convolution Layers Method \ Capacity x$\frac{1}{2}$ x$1$ x$2$
x$\frac{1}{2}$ x$1$ x$2$ x$\frac{1}{2}$ x$1$ x$2$ AE Central
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.524$\pm{}$0.035
0.515$\pm{}$0.042 0.560$\pm{}$0.043 0.517$\pm{}$0.040 0.416$\pm{}$0.038
0.469$\pm{}$0.041 0.389$\pm{}$0.033 0.565$\pm{}$0.040 0.395$\pm{}$0.034
DeepLIFT 0.818$\pm{}$0.032 0.826$\pm{}$0.029 *0.858$\pm{}$0.025
0.788$\pm{}$0.032 0.837$\pm{}$0.028 0.823$\pm{}$0.031 0.483$\pm{}$0.041
0.778$\pm{}$0.032 0.752$\pm{}$0.039 GradShap 0.711$\pm{}$0.032
0.699$\pm{}$0.036 0.668$\pm{}$0.037 0.678$\pm{}$0.034 0.670$\pm{}$0.033
0.700$\pm{}$0.034 0.567$\pm{}$0.035 0.676$\pm{}$0.034 0.737$\pm{}$0.030
Coalition $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$
0.341$\pm{}$0.026 0.357$\pm{}$0.023 0.325$\pm{}$0.025 0.307$\pm{}$0.028
0.341$\pm{}$0.027 0.324$\pm{}$0.029 0.328$\pm{}$0.027 0.331$\pm{}$0.026
0.339$\pm{}$0.029 BAE Central
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.510$\pm{}$0.035
0.565$\pm{}$0.039 0.586$\pm{}$0.040 0.492$\pm{}$0.041 0.469$\pm{}$0.038
0.532$\pm{}$0.042 0.360$\pm{}$0.030 0.523$\pm{}$0.039 0.432$\pm{}$0.034
$\mathrm{Var_{\theta}(NLL)}$ 0.574$\pm{}$0.038 0.749$\pm{}$0.030
0.816$\pm{}$0.026 0.728$\pm{}$0.034 0.727$\pm{}$0.030 0.715$\pm{}$0.029
0.579$\pm{}$0.037 0.732$\pm{}$0.031 0.712$\pm{}$0.036 Coalition
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.338$\pm{}$0.025
0.326$\pm{}$0.025 0.325$\pm{}$0.025 0.313$\pm{}$0.029 0.324$\pm{}$0.028
0.327$\pm{}$0.028 0.333$\pm{}$0.029 0.325$\pm{}$0.028 0.337$\pm{}$0.028
$\mathrm{Var_{\theta}(NLL)}$ 0.335$\pm{}$0.029 0.352$\pm{}$0.029
0.373$\pm{}$0.029 0.338$\pm{}$0.026 0.360$\pm{}$0.028 0.314$\pm{}$0.029
0.326$\pm{}$0.028 0.332$\pm{}$0.026 0.315$\pm{}$0.028

Table 7: Mean and standard error of absolute Matthew correlation coefficient
for ranking of sensors using various explanation methods with varied number of
convolution layers and model capacities evaluated on (a) PRONOSTIA and (b)
ZEMA datasets. Highest mean value of each column is bolded. * indicates
overall highest value.

(a) PRONOSTIA

|  |  | Matthew correlation coefficient
---|---|---|---
|  |  | 1 Convolution Layer | 2 Convolution Layers | 3 Convolution Layers
Method \ Capacity |  x$\frac{1}{2}$ |  x$1$ |  x$2$ |  x$\frac{1}{2}$ |  x$1$ |  x$2$ |  x$\frac{1}{2}$ |  x$1$ |  x$2$
AE | Central | $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ |  0.754$\pm{}$0.051 |  0.834$\pm{}$0.046 |  0.926$\pm{}$0.027 |  0.646$\pm{}$0.053 |  0.694$\pm{}$0.053 |  0.660$\pm{}$0.059 |  0.633$\pm{}$0.051 |  0.593$\pm{}$0.052 |  0.614$\pm{}$0.055
|  | DeepLIFT |  0.667$\pm{}$0.055 |  0.657$\pm{}$0.062 |  0.739$\pm{}$0.055 |  0.685$\pm{}$0.047 |  0.654$\pm{}$0.055 |  0.692$\pm{}$0.052 |  0.702$\pm{}$0.048 |  0.677$\pm{}$0.050 |  0.683$\pm{}$0.051
|  | GradShap |  0.934$\pm{}$0.027 |  0.898$\pm{}$0.033 |  0.868$\pm{}$0.046 |  0.816$\pm{}$0.048 |  0.819$\pm{}$0.052 |  0.831$\pm{}$0.044 |  0.701$\pm{}$0.059 |  0.719$\pm{}$0.061 |  0.814$\pm{}$0.050
| Coalition | $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ |  0.796$\pm{}$0.056 |  0.829$\pm{}$0.051 |  0.953$\pm{}$0.021 |  0.635$\pm{}$0.056 |  0.745$\pm{}$0.054 |  0.752$\pm{}$0.052 |  0.611$\pm{}$0.052 |  0.612$\pm{}$0.056 |  0.625$\pm{}$0.058
BAE | Central | $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ |  0.786$\pm{}$0.048 |  0.821$\pm{}$0.047 |  0.834$\pm{}$0.050 |  0.656$\pm{}$0.054 |  0.703$\pm{}$0.052 |  0.668$\pm{}$0.056 |  0.606$\pm{}$0.053 |  0.639$\pm{}$0.052 |  0.636$\pm{}$0.056
|  | $\mathrm{Var_{\theta}(NLL)}$ |  0.791$\pm{}$0.052 |  0.874$\pm{}$0.037 |  0.825$\pm{}$0.049 |  0.705$\pm{}$0.054 |  0.746$\pm{}$0.050 |  0.651$\pm{}$0.062 |  0.699$\pm{}$0.058 |  0.742$\pm{}$0.054 |  0.664$\pm{}$0.058
| Coalition | $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ |  0.816$\pm{}$0.049 |  0.897$\pm{}$0.039 |  *0.971$\pm{}$0.016 |  0.699$\pm{}$0.054 |  0.700$\pm{}$0.055 |  0.756$\pm{}$0.051 |  0.634$\pm{}$0.053 |  0.696$\pm{}$0.053 |  0.658$\pm{}$0.051
|  | $\mathrm{Var_{\theta}(NLL)}$ |  0.689$\pm{}$0.052 |  0.697$\pm{}$0.056 |  0.785$\pm{}$0.058 |  0.717$\pm{}$0.058 |  0.688$\pm{}$0.056 |  0.747$\pm{}$0.055 |  0.771$\pm{}$0.047 |  0.819$\pm{}$0.044 |  0.746$\pm{}$0.054

(b) ZEMA  Matthew correlation coefficient 1 Convolution Layer 2 Convolution
Layers 3 Convolution Layers Method \ Capacity x$\frac{1}{2}$ x$1$ x$2$
x$\frac{1}{2}$ x$1$ x$2$ x$\frac{1}{2}$ x$1$ x$2$ AE Central
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.473$\pm{}$0.043
0.462$\pm{}$0.045 0.482$\pm{}$0.046 0.480$\pm{}$0.044 0.459$\pm{}$0.047
0.456$\pm{}$0.045 0.481$\pm{}$0.048 0.455$\pm{}$0.046 0.428$\pm{}$0.045
DeepLIFT 0.399$\pm{}$0.045 0.333$\pm{}$0.045 0.327$\pm{}$0.045
0.388$\pm{}$0.044 0.396$\pm{}$0.045 0.330$\pm{}$0.044 0.310$\pm{}$0.040
0.332$\pm{}$0.044 0.371$\pm{}$0.044 GradShap 0.207$\pm{}$0.034
0.206$\pm{}$0.035 0.213$\pm{}$0.034 0.221$\pm{}$0.034 0.220$\pm{}$0.034
0.215$\pm{}$0.034 0.271$\pm{}$0.040 0.218$\pm{}$0.036 0.259$\pm{}$0.040
Coalition $\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$
0.505$\pm{}$0.050 0.551$\pm{}$0.046 0.516$\pm{}$0.050 0.488$\pm{}$0.050
0.505$\pm{}$0.048 0.493$\pm{}$0.049 0.502$\pm{}$0.049 0.499$\pm{}$0.049
0.487$\pm{}$0.048 BAE Central
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.471$\pm{}$0.043
0.483$\pm{}$0.045 0.482$\pm{}$0.046 0.452$\pm{}$0.046 0.448$\pm{}$0.045
0.456$\pm{}$0.046 0.403$\pm{}$0.050 0.446$\pm{}$0.046 0.448$\pm{}$0.046
$\mathrm{Var_{\theta}(NLL)}$ 0.422$\pm{}$0.045 0.441$\pm{}$0.043
0.454$\pm{}$0.045 0.457$\pm{}$0.048 0.422$\pm{}$0.043 0.454$\pm{}$0.047
0.401$\pm{}$0.047 0.449$\pm{}$0.048 0.440$\pm{}$0.048 Coalition
$\operatorname*{\mathbb{E}}_{\theta}\mathrm{(NLL)}$ 0.510$\pm{}$0.050
0.554$\pm{}$0.047 0.512$\pm{}$0.050 0.488$\pm{}$0.050 0.488$\pm{}$0.050
0.492$\pm{}$0.049 0.463$\pm{}$0.049 0.491$\pm{}$0.050 0.485$\pm{}$0.049
$\mathrm{Var_{\theta}(NLL)}$ 0.654$\pm{}$0.039 0.378$\pm{}$0.046
0.384$\pm{}$0.048 0.579$\pm{}$0.051 *0.660$\pm{}$0.036 0.406$\pm{}$0.050
0.420$\pm{}$0.046 0.553$\pm{}$0.048 0.408$\pm{}$0.048

(a) Pearson correlation coefficient

(b) Matthew correlation coefficient

Figure 9: Critical difference diagrams for comparing explanation methods under
metrics of (a) Pearson correlation coefficient between $S^{shift}$ and
$S^{\bot{shift}}$ attribution scores and (b) Matthew correlation coefficient
for ranking of sensors. Average rank of each method is shown next to the
labels. The methods on the right have higher ranks than the methods on left. A
thick line connecting the methods indicate there is no significant difference
between the methods.