# A Robust Hot Subdwarfs Identification Method Based on Deep Learning

Lei Tan Center For Astrophysics, Guangzhou University,Guangzhou, Guangdong,
China, 510006  Great Bay Center, National Astronomical Data Center, Guangzhou,
Guangdong, China, 510006  Ying Mei Center For Astrophysics, Guangzhou
University,Guangzhou, Guangdong, China, 510006  Great Bay Center, National
Astronomical Data Center, Guangzhou, Guangdong, China, 510006  Zhicun Liu CAS
Key Laboratory of Optical Astronomy, National Astronomical Observatories,
Chinese Academy of Sciences, Beijing, China, 100101  University of Chinese
Academy of Sciences, Beijing, China, 100049  Yangping Luo Department of
Astronomy, China West Normal University, Nanchong, China, 637002  Hui Deng
Center For Astrophysics, Guangzhou University,Guangzhou, Guangdong, China,
510006  Great Bay Center, National Astronomical Data Center, Guangzhou,
Guangdong, China, 510006  Feng Wang Center For Astrophysics, Guangzhou
University,Guangzhou, Guangdong, China, 510006  Great Bay Center, National
Astronomical Data Center, Guangzhou, Guangdong, China, 510006  University of
Chinese Academy of Sciences,Beijing, China, 100049  Linhua Deng  Yunnan
Observatories, Chinese Academy of Sciences, Kunming, China, 650011  University
of Chinese Academy of Sciences, Beijing, China, 100049  Chao Liu Key
Laboratory of Space Astronomy and Technology, National Astronomical
Observatories, Chinese Academy of Sciences, Beijing, China, 100101  University
of Chinese Academy of Sciences, Beijing, China, 100049

###### Abstract

Hot subdwarf star is a particular type of star that is crucial for studying
binary evolution and atmospheric diffusion processes. In recent years,
identifying Hot subdwarfs by machine learning methods has become a hot topic,
but there are still limitations in automation and accuracy. In this paper, we
proposed a robust identification method based on the convolutional neural
network (CNN). We first constructed the dataset using the spectral data of
LAMOS DR7-V1. We then constructed a hybrid recognition model including an
8-class classification model and a binary classification model. The model
achieved an accuracy of 96.17% on the testing set. To further validate the
accuracy of the model, we selected 835 Hot subdwarfs that were not involved in
the training process from the identified LAMOST catalog (2428, including
repeated observations) as the validation set. An accuracy of 96.05% was
achieved. On this basis, we used the model to filter and classify all
10,640,255 spectra of LAMOST DR7-V1, and obtained a catalog of 2393 Hot
subdwarf candidates, of which 2067 have been confirmed. We found 25 new Hot
subdwarfs among the remaining candidates by manual validation. The overall
accuracy of the model is 87.42%. Overall, the model presented in this study
can effectively identify specific spectra with robust results and high
accuracy, and can be further applied to the classification of large-scale
spectra and the search of specific targets.

LAMOST, Hot subdwarf, Spectral Classification, Convolutional Neural Network

##  1 Introduction

Hot subdwarfs are core helium-burning stars located below the upper main
sequence of the Hertzsprung-Russell diagram (HRD) and referred to as extreme
horizontal branch (EHB) stars because of their evolution stage. In general,
Hot subdwarfs are classified into three types by their spectra, i.e., subdwarf
B (sdB), subdwarf O (sdO), and subdwarf OB (sdOB).

The study of Hot subdwarfs has significant scientific value. O’Connell (1999)
and Han et al. (2007) presented that the Hot subdwarfs are the primary sources
of the ultraviolet (UV) upturn phenomena found in elliptical galaxies. They
are also critical studies involving close binary interactions because most sdB
stars are formed in binary systems. Geier et al. (2007) presented that Hot
subdwarf binaries with massive White dwarf companions are believed to be the
precursors of type Ia supernova. The peculiar atmospheres of Hot subdwarfs can
be used to study gravitational settling and radiative levitation (O’Toole &
Heber 2006, Geier 2013). Hot subdwarfs with pulsations can be used for
asteroseismic analyses (Charpinet et al. 2011). Lei et al. (2015, 2016) used
Hot subdwarfs to study the extended horizontal branch morphology of globular
clusters.

Driven by scientific research, searching and identifying Hot subdwarfs and
constructing Hot subdwarfs catalogs have become a hot direction in Hot
subdwarfs research. The traditional method of searching subdwarfs is mainly
based on the basic characteristics of Hot subdwarfs. For example, Vennes et
al. (2011) identified 48 Hot subdwarfs based on UV photometry of the GALEX
survey. Luo et al. (2016) identified 166 Hot subdwarfs using photometric
information. Geier et al. (2019) used color, absolute magnitude, and reduced
motion cut methods to compile a candidate catalog of 39,800 Hot subdwarfs.
However, these identification methods mainly rely on manual processing, which
is laborious and difficult to meet the demands of handling large-scale
spectral data.

The Large Area Multi-Objective Fiber Optic Spectroscopic Telescope
(LAMOST)(Cui et al., 2012; Zhao et al., 2012; Deng et al., 2012) is a Schmidt
telescope with an effective aperture of 3.6$-$4.9m with the field of view of
about 5∘. Being a survey telescope, the LAMOST was designed to collect 4000
spectra in a single exposure (spectral resolution R$\sim$1800, limiting
magnitude r$\sim$19 mag, wavelength coverage 3700Å$-$9000Å). The LAMOST DR7-V1
catalog contains 10,640,255 calibrated spectra. The total catalogue includes
9,881,260 STAR spectra, 198,393 GALAXY spectra, 66,406 QSO spectra and 494,196
spectra of Unknown-type.

Identifying Hot subdwarfs from the LAMOST catalog has essential research value
because the LAMOST can give the spectral characteristics of Hot subdwarfs that
reveal details of the formation and evolution of Hot subdwarfs. However,
identifying Hot subdwarfs from LAMOST data is a challenging task. The critical
reason is that the LAMOST does not have homogeneous color data (Luo et al.
2015) and therefore traditional methods such as color-cut (Bu et al. 2019)
cannot be used to search for Hot subdwarfs in the LAMOST catalog. Another
reason is the massive amount of catalog data. Scientists have experimented
with other methods to identify HOT subdwarfs from the LAMOST catalog. Luo et
al. (2021) identified 1587 Hot subdwarfs using the spectra released by the
LAMOST and the catalog of Hot subdwarf candidates released by Gaia DR2. With
the development of machine learning techniques, especially the deep learning
techniques, Bu et al. (2017) used machine learning methods to search for Hot
subdwarf candidates from the LAMOST DR1 and obtained 10,000 candidates, and
Lei et al. (2019) further identified 56 Hot subdwarfs from these candidates.
Bu et al. (2019) applied Convolutional Neural Network (CNN) to construct a
binary classification model for Hot subdwarfs searching in LAMOST DR4 and
achieved an F1 value (defined in Section 3.3) of 76.98$\%$.

Overall, these artificial intelligence-based identification methods have
achieved remarkable results. Identifying Hot subdwarf based on deep learning
can significantly reduce the difficulty of manual identification and obtain
credible results. It can be considered one of the most effective methods to
search for a specific target in a large amount of catalog data.

However, there are still some minor deficiencies in current research on deep
learning-based recognition, mainly focusing on the imbalance of data samples.
In the model of Bu et al. (2019), 510 Hot subdwarfs and 5458 other classes
(star, galaxy, unknown object, and so on) were used for the binary
classification model. From a practical point of view, this ratio is grossly
unbalanced. Studies have shown that the unbalanced number of datasets for
different classes can affect the reliability and accuracy of the
classification results. Insufficient sample size can lead to overfitting of
the model (Zhan et al. 2018; Zhu et al. 2018). At the same time, the model
will be incompleteness due to insufficient sample size (Du et al. 2016).

In light of the previous literature, a hybrid model with an 8-class
classification model and a binary classification model is proposed and applied
to search for Hot subdwarfs in the LAMOST catalog. We introduce the data and
preprocessing in Section 2, including data preparation and data augmentation.
In Section 3, we describe the construction process of our classification model
and give a detailed performance evaluation. In Section 4, the model is applied
to classify the entire LAMOST spectral data to obtain Hot subdwarf candidates.
The candidates are identified by manual authentication. The ability of our
model is further discussed in Section 5. In Section 6, we summarize the work
and propose some future work.

##  2 Data and Preprocessing

###  2.1 Sample set

Based on the LAMOST DR7-V1 catalog, we created a sample set that extracted the
data with all spectra classes (O, Hot subdwarf, White dwarf, B, A, F, G, K,
and M) to meet the labeling requirements of the deep learning techniques.
Referring to the sample selection criteria of Bu et al. (2019), we selected
samples with SNR greater than 10 to construct the sample set. When the SNR was
set to be greater than 10, there were sufficient samples for Hot subdwarf,
White dwarf, B, A, F, G, K, and M stars, while there were only 72 O-type
stars.

###  2.2 Data Augmentation

To ensure the balance of the sample, we have to augment O-type stars according
to the characteristics of the existing 72 samples. The spectrum of a star is a
blackbody radiation spectrum, which is accompanied by emission and absorption
lines. That is to say, the generation of a spectrum needs to satisfy
characteristics of both two aspects, such as blackbody radiation spectrum and
emission/absorption lines.

1\. Blackbody radiation spectrum fitting

The blackbody radiation spectrum of a star is defined by the Planck formula
(Equation 1, a relationship between the flux, the wavelength ($\lambda$) and
the temperature ($t$) of the spectrum). To further compensate for the effects
of relative flux correction and sky background, we modify the formula to
Equation 2, where $a$ compensates for effects of relative flux correction, and
$b$ is compensated for effects of the skylight background.

$flux(\lambda,t)=\frac{8\pi hc}{\lambda^{5}}\cdot\frac{1}{e^{\frac{hc}{\lambda
kt}}-1}$ (1)

$flux(\lambda,t,a,b)=a\cdot\frac{8\pi
hc}{\lambda^{5}}\cdot\frac{1}{e^{\frac{hc}{\lambda kt}}-1}+b$ (2)

In order to fit the temperature $t$, $a$ and $b$ in Equation 2, in practice,
we obtained 3700 flux values with wavelengths from $3700\AA$ to $8672\AA$ by
truncating each O-type star spectrum. The emission and absorption lines were
removed by median filtering, and the corresponding blackbody radiation was
obtained. The fitting process can be realized by the
scipy.optimize.curve_fit111https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html
function in the Python Scipy package. For each known O-type star spectrum,
optimal values were obtained for the parameters ($t$, $a$, $b$) so that the
sum of the squared residuals of $flux_{i}-{flux}^{\prime}_{i}$ was minimized
(i.e., Equation 3 was minimized). In Equation 3, $flux$ is the original
blackbody radiation spectrum and ${flux}^{\prime}$ is the fitted flux value
obtained by substituting the fitted parameters into Equation 2.

$\alpha=\sum_{i=1}^{3700}(flux_{i}-{flux}^{\prime}_{i})^{2}$ (3)

Consequently, 72 sets of parameters (t, a, b) were obtained by fitting 72
O-type samples. Figure 1 shows the real blackbody radiation spectrum and the
generated blackbody radiation spectrum, proving consistent in their physical
characteristics.

2\. Obtain of emission and absorption lines

Considering that the emission and absorption lines for a specific class are
basically the same, the emission and absorption lines of O-type stars can be
obtained by subtracting the blackbody radiation spectrum (obtained by median
filtering) from the original spectrum. For the 72 known O-type stars, 72 sets
of emission and absorption lines can finally be obtained.

To generate a new blackbody radiation spectrum of the O-type star, we set a
random temperature (between 28,000K and 100,000K) and pick a random set of
parameters $a$, $b$ to obtain the flux calculating by Equation 2. The final
generated spectrum was obtained by randomly combining the fitted blackbody
radiation spectrum with the emission and absorption lines. Algorithm
References lists the spectrum generation steps in pseudo-code format.From the
model training results in section 3.3, the generated data can solve the
overfitting problem caused by the insufficient amount of data well and get a
good accuracy rate at the same time, which meets the requirements of deep
learning.

(a) $\alpha$ = 74

(b) $\alpha$ = 146

(c) $\alpha$ = 502

Figure 1: The real blackbody radiation spectrum ($flux_{i}$, the blue curve)
and the generated blackbody radiation spectrum ($flux^{\prime}_{i}$, the red
curve) under the unified observational conditions (temperature $t$, flux
correction $a$ and the sky background $b$). Three samples are given here. The
first two are cases where the sum of the squared residuals are relatively
small, and sample (c) is a poor fit.

##  3 Spectral Classification Model Based on CNN

###  3.1 Model Design

In this section, a CNN-based model for classifying the spectra is proposed. A
straightforward idea of model construction is to classify all the spectra into
9 classes and get different types of candidates. However, the preliminary
experiments proved that the probability of confusing Hot subdwarfs and White
dwarfs in the 9-classification model is high due to their similar blackbody
radiation characteristics. Therefore, a more refined hybrid model with an
8-class classification model and a binary classification model is constructed.
The 8-class classification model disregards the effect of White dwarfs, and
the binary classification model further separates Hot subdwarfs from White
dwarfs.

The network consists of 6 convolutional layers. A max-pooling layer follows
each convolutional layer. After the convolutional and max-pooling layers, two
fully connected layers are added. In the convolutional layer, we process the
output data by the ReLU function (Nair & Hinton 2010). Cross-entropy function
(Hinton & Salakhutdinov 2006) is used as the loss function, and SGD (Hardt et
al. 2015) is used as the optimizer. During training, we start with small
convolutional kernels and gradually increase the size of the kernels, and the
final parameters of each layer are given in Table LABEL:modt. The Softmax
function (Ian et al. 2016) gives the final classification results, which are
probability values of the samples being classified into different classes.

Figure 2: The architecture of the classification model. The two rows of numbers under the model represent the feature map size, the top one is for 8-class classification model and the bottom one is for binary classification model. Table 1: Parameters of each layer in the model. Layer | Type | Maps | Kernel Size | Striding | Activation
---|---|---|---|---|---
IN | INPUT | 1 |  $\cdots$ |  $\cdots$ |  $\cdots$
C1 | Convolution | 10 | 1$\times$2 | 1 | ReLU
S2 | Max Pooling | 10 | 1$\times$2 | 1 |  $\cdots$
C3 | Convolution | 20 | 1$\times$3 | 1 | ReLU
S4 | Max Pooling | 20 | 1$\times$2 | 1 |  $\cdots$
C5 | Convolution | 30 | 1$\times$4 | 1 | ReLU
S6 | Max Pooling | 30 | 1$\times$2 | 2 |  $\cdots$
S7 | Convolution | 40 | 1$\times$5 | 1 | ReLU
S8 | Max Pooling | 40 | 1$\times$2 | 1 |  $\cdots$
C9 | Convolution | 50 | 1$\times$7 | 1 | ReLU
S10 | Max Pooling | 50 | 1$\times$2 | 2 |  $\cdots$
C11 | Convolution | 60 | 1$\times$9 | 1 | ReLU
S12 | Max Pooling | 60 | 1$\times$2 | 2 |  $\cdots$
F13 | Fully Connected |  $\cdots$ |  $\cdots$ |  $\cdots$ |  $\cdots$
F14 | Fully Connected |  $\cdots$ |  $\cdots$ |  $\cdots$ |  $\cdots$
OUT | Fully Connected |  $\cdots$ |  $\cdots$ |  $\cdots$ | Softmax

###  3.2 Data set

After sample augmentation of the O-type star spectrum, we randomly selected
data and performed manual validation until there were 2000 samples for each
class (O, Hot subdwarf, White dwarf, B, A, F, G, K, and M stars, a total of
18,000 samples). 70$\%$ of the samples were randomly selected as the training
set, and 30$\%$ were used for testing. Considering that our goal was to
construct a model for searching Hot subdwarfs, we did not include White dwarfs
in the training and testing set of the 8-class classification model. 3700 flux
values in wavelength range of 3700Å to 8672Å were obtained by truncating each
sample. Specifically, the LAMOST DR7-V1 provides spectral data with a
wavelength increment of $\lg(\lambda_{i+1})-\lg(\lambda_{i})=0.0001$,
therefore there are 3700 data points sampled in the wavelength range of 3700Å
to 8672Å . We did not resample the original spectral data in our analysis.

The binary classification model was constructed to distinguish Hot subdwarfs
from White dwarfs accurately. Since the blackbody radiation spectrum of Hot
subdwarf and White dwarf are very similar, it is difficult for the model to
distinguish them when trained with the original spectra. Therefore, for the
binary classification model, the emission and absorption lines of the spectra
were used for training and testing. Experiments showed that the model fit best
when trained with the 1400 flux values (the emission and absorption lines)
corresponding to wavelengths between 3700Å to 5106Å for each spectrum. 835 Hot
subdwarfs with SNR greater than 10 and not involved in the training process
were used to further validate the model. The 8-class classification model and
binary classification model were trained using the same Hot subdwarf training
set to obtain a more robust model. All of the samples were normalized by
Equation 4 to prevent gradient problems during deep learning model training,
where $x_{min}$ was the minimum value of the flux and $x_{max}$ was the
maximum value of the flux.

${x}^{\prime}=\frac{x-x_{min}}{x_{max}-x_{min}}$ (4)

###  3.3 Modelling and Performance Analysis

The performance of a deep learning model is usually evaluated by accuracy,
precision, recall, and F1 score (Forman et al. 2003), which are parameters
calculated from the confusion matrix. A confusion matrix includes four parts,
True Positive (TP), False Positive (FP), True Negative (TN), and False
Negative (FN). TP and TN are the observations that are correctly predicted. FP
and FN are cases of misclassification. Accordingly, the definition of
accuracy, precision, recall and F1 score are in Equation 5, Equation 6,
Equation 7 and Equation 8.

For the 8-class classification model, we set the batch size to 100 and the
number of iterations to 2500. After training, we found that the model
gradually fitted after about 2200 iterations. For the binary classification
model, we set the batch size to 10 and the number of iterations to 2000. The
model gradually fitted after about 1750 iterations. Through repeated training
and testing, we finally got an accuracy of 94.21$\%$ for the 8-class
classification model and accuracy of 96.17$\%$ for the binary classification
model. The confusion matrixes are shown in Figure 3. The precision, recall,
and F1 score for each class are presented in Table 2 and Table 3. We also
output the mean $\epsilon$ and variance $\mu$ of the difference between the
predicted and true labels of the two models. For the 8-class classification
model, the mean $\epsilon$ is 0.0166 and the variance $\mu$ is 0.67. For the
binary classification model, the mean $\epsilon$ is 0.023 and the variance
$\mu$ is 0.46. The results show that the model performs well, with a tiny
difference between the predicted and true labels.

$Accuracy=\frac{TP+TN}{TP+FP+FN+TN}$ (5)

$Precision=\frac{TP}{TP+FP}$ (6)

$Recall=\frac{TP}{TP+FN}$ (7)

$F1=\frac{2*Precision*Recall}{Precision+Recall}$ (8)

(a) 8-class classification model

(b) Binary classification model

Figure 3: The confusion matrix. Subfigure (a) is the confusion matrix of the 8-class classification model and subfigure (b) is the confusion matrix of the binary classification model. Table 2: Precision, Recall and F1 score of the 8-class classification model for each class. Class | Precision | Recall | F1
---|---|---|---
O | 98.98$\%$ | 99.32 $\%$ | 99.15$\%$
H | 98.48$\%$ | 97.82 $\%$ | 98.15$\%$
B | 95.41$\%$ | 94.44 $\%$ | 94.92$\%$
A | 88.25$\%$ | 92.85 $\%$ | 90.49$\%$
F | 88.97$\%$ | 89.43 $\%$ | 89.20$\%$
G | 92.23$\%$ | 88.93 $\%$ | 90.55$\%$
K | 95.26$\%$ | 92.45 $\%$ | 93.83$\%$
M | 96.47$\%$ | 98.53 $\%$ | 97.49$\%$
Table 3: Precision, Recall and F1 score of the binary classification model for each class. Class | Precision | Recall | F1
---|---|---|---
Hot subdwarf | 96.45$\%$ | 95.81 $\%$ | 96.13$\%$
White dwarf | 95.89$\%$ | 96.52 $\%$ | 96.20$\%$

To further test the ability of our model for searching Hot subdwarfs, we used
the 835 identified Hot subdwarfs which were not involved in the training
process for validation. The classification results showed that 802 Hot
subdwarfs (96.05$\%$ of the total) were correctly identified. An analysis and
discussion of the misclassified samples are given in Section 5.

##  4 Hot subdwarfs Identification and manual Validation

We identified all spectra in the LAMOST DR7 using the hybrid CNN model
implemented in the previous section. The Hot subdwarf candidates obtained by
the model were verified manually and a final credible catalog was given.

The classification results of the model gave the categories and the
corresponding probabilities for all the spectrum of LAMSOT. By analysing
prediction probabilities of the 2235 identified Hot subdwarfs with SNR greater
than 10, we found that the model gave a prediction probability of more than
80% for 2103 of them, wherein the predicted probability of 2043 were more than
99%. With reference to the work of Zheng et al. (2020), we considered samples
with a prediction probability above 99% to be more reliable.

In order to obtain reliable results, we did further screening of the results
of model predictions from two aspects: (a) For candidates with SNR greater
than 10 ($g$-band), we retained the recognized spectra with probability higher
than 99%. (b) For candidates with SNR lower than 10 ($g$-band), we selected
the top 200 samples with the highest prediction probabilities. Finally we
obtained 2393 Hot subdwarf candidates.

We checked the obtained candidates by cross-validating with the catalog of Hot
subdwarfs published by Luo et al. (2021) and Geier et al. (2017) and manually
validating. Manual validation results of the 2393 Hot subdwarf candidates are
presented in Table 4. Along with basic information of the spectra, the last
column of Table 4 describes the manual validation results. The results of
manual validation are finally summarized into four categories: 1) Hot
subdwarfs that have been identified by previous work, 2) Newly discovered Hot
subdwarf stars, 3) Identified as other classes by manual validation, 4) Cannot
determine the class due to low SNR. The results show that among the 2393
candidates, 2092 are Hot subdwarfs (25 are newly discovered), 183 are other
types of spectra and misclassified as Hot subdwarfs, and 118 spectra have low
SNR and cannot be manually confirmed. In other words, 87.42$\%$ of the Hot
subdwarfs predicted by the model are proved to be Hot subdwarfs in artificial
verification.

Following the works of Luo et al. (2016) and Luo et al. (2021), atmospheric
parameters (Effective temperature Teff, surface gravity $log$(g), and $He$
abundances $log(nHe/nH))$ for the 25 newly identified Hot subdwarf stars were
obtained by fitting the LAMOST spectra with TLUSTY/SYNSPEC non-LTE synthetic
spectra (Hubeny & Lanz 1995, 2011). All the parameters of the 25 newly
discovered Hot subdwarfs are listed in Table 5, which includes the LAMOST
designation, right ascension, declination, effective temperature, surface
gravity, $He$ abundance, the SNR in the $\mu$-band, apparent magnitudes in the
$\mu$-band and $g$-band of SDSS DR9, and apparent magnitudes in the $g$-band
of Gaia DR2.

##  5 Discussions

###  5.1 Model performance

Compared with the previous binary classification model (Hot subdwarfs and
other types), a more accurate hybrid deep learning model is constructed in
this paper. It achieved an accuracy rate of 96.17% on the testing set and 802
of the 835 Hot subdwarfs (96.05%) in the validation sets are correctly
identified. From Table 3, we can see that the F1 value for searching Hot
subdwarfs is 96.13$\%$, which is much higher than the 76.98$\%$ in Bu et al.
(2019).

Comparative analysis shows that the overfitting problem caused by insufficient
data can be solved by data augmentation. The improvement of training samples
also enables the model to learn more complete features, resulting in better
results. Overall, the model structure that combined 8-class classification
with binary classification model obtains better performance than the binary
classification model alone.

###  5.2 Misclassification

By analyzing the classification results in Section 4, it can be found that for
2428 Hot subdwarfs in the LAMOST catalog, the model eventually found 2067.
Therefore, we analyzed the remaining 361 Hot subdwarfs. Through sample
analysis, it can be concluded that: (1) 89 Hot subdwarfs were misclassified by
our model, (2) 272 Hot subdwarfs were predicted as Hot subdwarfs by our model
with a low probability and filtered out during candidate screening (in Section
4).

The spectral characteristics of most correctly classified Hot subdwarfs are
consistent with Figure 4a. We performed a detailed analysis of the
misclassified spectra and the low probability spectra, and it was found that
there are mainly three conditions: (1) Low SNR data (Figure 4b). It is
difficult to identify the absorption and emission lines. For the effect of
noise, our experiments find that the prediction probability gradually
decreases as the noise increases. (2) Spectrum with abnormal value, that is,
spectrum dominated by sharp spikes either from cosmic rays or instrumental
artifacts (Figure 4c). We found through experiments that by artificially
setting abnormal values for Hot subdwarfs filtered out by our models can also
lead to misclassification and low prediction probability. (3) A spectrum
exhibits the features of a binary star, as seen in Figure 4d, which is
probably a binary star (possibly a Hot subdwarf and a M star). To improve the
classification accuracy, we need more labeled training data. However, there
are very few samples of these types in the LAMOST catalog. These three factors
lead to a sample being misclassified or classified with a very low probability
as a Hot subdwarf.

For spectrum with low SNR and the category cannot be manually confirmed (Table
4, type 4), repeat observations of the LAMOST can be continuously followed at
a later stage.

(a) spectrum of most Hot subdwarfs

(b) low SNR spectrum

(c) Spectrum with abnormal value

(d) A spectrum exhibits the features of a binary star

Figure 4: Examples of Hot subdwarfs that were misclassified.

##  6 Conclusions

In this study, our proposed approach uses a hybrid CNN-based model to classify
the spectra and search for Hot subdwarfs from the LAMOST data. For the problem
of insufficient data during model training, we propose a spectral generation
method based on fitting the Planck formula. The model is applied to identify
existing Hot subdwarfs in the LAMOST catalog, and most of the Hot Subdwarfs
are correctly identified by our model. Moreover, our approach offers superior
performance to the previous binary classification approach. The proposed model
is finally used to classify the LAMOST data and search for Hot subdwarfs. The
experiments indicate that 87.42% of the Hot subdwarfs predicted by our model
are proved to be Hot subdwarfs. The 25 newly identified Hot subdwarfs would
provide a reference for follow-up researches in related fields.

In addition to newly discovered Hot subdwarfs, the current approach is well
suited for searching specific targets in vast amounts of spectral data. In
particular, spectrum generation provides a solution for samples with a small
amount of data and guarantees the reliability of deep learning methods in
spectrum classification. Improving the sample size is one of the following
efforts, which is essential for improving the model accuracy. We believe that
the multi-classification model implemented in this paper is up-and-coming to
be successfully applied to search for other specific stars. Capability and
performance of the model are worthy of further experiments. Codes of the
spectral generation algorithm, model construction, and model training are
available at https://github.com/astronomical-data-
processing/cnn_spectra_classification.

##  7 Acknowledgments

This work is supported by the National SKA Program of China (2020SKA0110300),
the National Science Foundation for Young Scholars (11903009), the Joint
Research Fund in Astronomy (U1831204, U1931141) under cooperative agreement
between the National Natural Science Foundation of China (NSFC) and the
Chinese Academy of Sciences (CAS), Funds for International Cooperation and
Exchange of the National Natural Science Foundation of China (11961141001),
the National Science Foundation of China (12173028), Fundamental and
Application Research Project of Guangzhou (202102020677), the Innovation
Research for the Postgraduates of Guangzhou University under Grant
(2021GDJC-M15).

We thank the anonymous referee for valuable and helpful comments and
suggestions.

## References

  * Bu et al. (2017) Bu, Y., Lei, Z., Zhao, G., Bu, J., & Pan, J. 2017, ApJS, 233, 2, doi: 10.3847/1538-4365/aa91cd
  * Bu et al. (2019) Bu, Y., Zeng, J., Lei, Z., & Yi, Z. 2019, ApJ, 886, 128, doi: 10.3847/1538-4357/ab4c47
  * Charpinet et al. (2011) Charpinet, S., Van Grootel, V., Fontaine, G., et al. 2011, A&A, 530, A3, doi: 10.1051/0004-6361/201016412
  * Cui et al. (2012) Cui, X.-Q., Zhao, Y.-H., Chu, Y.-Q., et al. 2012, RAA, 12, 1197
  * Deng et al. (2012) Deng, L.-C., Newberg, H. J., Liu, C., et al. 2012, Research in Astronomy and Astrophysics, 12, 735, doi: 10.1088/1674-4527/12/7/003
  * Du et al. (2016) Du, C., Luo, A., Yang, H., Hou, W., & Guo, Y. 2016, PASP, 128, 034502, doi: 10.1088/1538-3873/128/961/034502
  * Forman et al. (2003) Forman, G., et al. 2003, J. Mach. Learn. Res., 3, 1289
  * Geier (2013) Geier, S. 2013, A&A, 549, A110, doi: 10.1051/0004-6361/201220549
  * Geier et al. (2007) Geier, S., Nesslinger, S., Heber, U., et al. 2007, A&A, 464, 299, doi: 10.1051/0004-6361:20066098
  * Geier et al. (2017) Geier, S., Østensen, R. H., Nemeth, P., et al. 2017, A&A, 600, A50, doi: 10.1051/0004-6361/201630135
  * Geier et al. (2019) Geier, S., Raddi, R., Gentile Fusillo, N. P., & Marsh, T. R. 2019, A&A, 621, A38, doi: 10.1051/0004-6361/201834236
  * Han et al. (2007) Han, Z., Podsiadlowski, P., & Lynas-Gray, A. E. 2007, MNRAS, 380, 1098, doi: 10.1111/j.1365-2966.2007.12151.x
  * Hardt et al. (2015) Hardt, M., Recht, B., & Singer, Y. 2015, arXiv e-prints, arXiv:1509.01240.  https://arxiv.org/abs/1509.01240
  * Hinton & Salakhutdinov (2006) Hinton, G. E., & Salakhutdinov, R. R. 2006, Science, 313, 504, doi: 10.1126/science.1127647
  * Hubeny & Lanz (1995) Hubeny, I., & Lanz, T. 1995, ApJ, 439, 875, doi: 10.1086/175226
  * Hubeny & Lanz (2011) —. 2011, Synspec: General Spectrum Synthesis Program.  http://ascl.net/1109.022
  * Ian et al. (2016) Ian, G., Yoshua, B., & Aaron, C. 2016, Deep Learning (Adaptive Computation and Machine Learning Series), The MIT Press
  * Lei et al. (2019) Lei, Z., Bu, Y., Zhao, J., Németh, P., & Zhao, G. 2019, PASJ, 71, 41, doi: 10.1093/pasj/psz006
  * Lei et al. (2015) Lei, Z., Chen, X., Zhang, F., & Han, Z. 2015, MNRAS, 449, 2741, doi: 10.1093/mnras/stv544
  * Lei et al. (2016) Lei, Z., Zhao, G., Zeng, A., et al. 2016, MNRAS, 463, 3449, doi: 10.1093/mnras/stw2242
  * Luo et al. (2015) Luo, A. L., Zhao, Y.-H., Zhao, G., et al. 2015, Research in Astronomy and Astrophysics, 15, 1095, doi: 10.1088/1674-4527/15/8/002
  * Luo et al. (2021) Luo, Y., Németh, P., Wang, K., Wang, X., & Han, Z. 2021, 256, 28, doi: 10.3847/1538-4365/ac11f6
  * Luo et al. (2016) Luo, Y.-P., Németh, P., Liu, C., Deng, L.-C., & Han, Z.-W. 2016, ApJ, 818, 202, doi: 10.3847/0004-637X/818/2/202
  * Nair & Hinton (2010) Nair, V., & Hinton, G. E. 2010, in Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML’10 (Madison, WI, USA: Omnipress), 807–814
  * O’Connell (1999) O’Connell, R. W. 1999, ARA&A, 37, 603, doi: 10.1146/annurev.astro.37.1.603
  * O’Toole & Heber (2006) O’Toole, S. J., & Heber, U. 2006, A&A, 452, 579, doi: 10.1051/0004-6361:20053948
  * Vennes et al. (2011) Vennes, S., Kawka, A., & Németh, P. 2011, MNRAS, 410, 2095, doi: 10.1111/j.1365-2966.2010.17584.x
  * Zhan et al. (2018) Zhan, Y., Hu, D., Wang, Y., & Yu, X. 2018, IEEE Geoscience and Remote Sensing Letters, 15, 212, doi: 10.1109/LGRS.2017.2780890
  * Zhao et al. (2012) Zhao, G., Zhao, Y.-H., Chu, Y.-Q., Jing, Y.-P., & Deng, L.-C. 2012, RAA, 12, 723, doi: 10.1088/1674-4527/12/7/002
  * Zheng et al. (2020) Zheng, Z.-P., Qiu, B., Luo, A. L., & Li, Y.-B. 2020, PASP, 132, 024504, doi: 10.1088/1538-3873/ab5ed7
  * Zhu et al. (2018) Zhu, L., Chen, Y., Ghamisi, P., & Benediktsson, J. A. 2018, IEEE Transactions on Geoscience and Remote Sensing, 56, 5046, doi: 10.1109/TGRS.2018.2805286

[h] : O-type spectrum generation

1:

  * •

$Sepc\\_original$: 72 O-type samples, each sepctrum contains 3700 flux values
at wavelength from 3700Å to 8672Å

  * •

$T\\_otype$: Temperature of O-type star, $T\\_otype\in[28,000K,100,000K]$

  * •

Modified Planck formula (Equation 2): $flux(\lambda,t,a,b)=a\cdot\frac{8\pi
hc}{\lambda^{5}}\cdot\frac{1}{e^{\frac{hc}{\lambda kt}}-1}+b$

2:

  * •

Generated O-type spectrum, $Sepc\\_generation$

3:for each spectrum ($Sepc\\_original_{i}$), $i\in[1,72]$ do

4: /*Step 1: blackbody radiation fitting*/

5: 1) Perform median filtering on the spectrum and obtain the blackbody
radiation spectrum, $flux_{i}$;

6: 2) According to the modified Planck formula, fit and minimize the value of
the objective function $\sum_{i=1}^{3700}(flux_{i}-{flux}^{\prime}_{i})^{2}$;

7: 3) Obtain the corresponding value of the parameters, temperature ($t_{i}$),
$a_{i}$ and $b_{i}$.

8:

9: /*Step 2: acquisition of absorption and emission lines*/

10: 4) Obtain the emission and absorption line:
$em\\_ab\\_line_{i}=sepc\\_original_{i}-flux_{i}$.

11:end for

12:

13:for $i\in[1,N(number\\_of\\_spectrum\\_to\\_be\\_generated)]$ do

14: /*Step 3: O-type spectrum generation*/

15: 1) Randomly choose a temperature ($T\\_otype_{k}$) between 28,000K and
100,000K;

16: 2) Randomly choose a set of a and b obtained in Step 1, and feed a, b and
$T\\_otype_{k}$ to Equation 2 (where $t$ = $T\\_otype_{k}$) to generate the
blackbody radiation spectrum, $spec\\_blackbody\\_gen_{k}$;

17: 3) Randomly choose a set of emission and absorption line obtained
($em\\_ab\\_line_{i}$) in Step 2;

18: 4) Generate O-type spectrum: $Sepc\\_generation$ =
$spec\\_blackbody\\_gen_{k}+em\\_ab\\_line_{i}$

19:end for

Table 4: Manual verification results of the 2393 Hot subdwarf candidates.

Designation | OBSID | RA∘‡ | Dec∘ | SNR | SNR | SNR | SNR | SNR |
---|---|---|---|---|---|---|---|---|---
LAMOST | LAMOST | LAMOST | LAMOST |  $g$-band |  $\mu$-band |  $z$-band |  $r$-band |  $i$-band | Type
J023200.24+333436.1 | 632206097 | 38.001041 | 33.576702 | 60.56 | 21.51 | 25.19 | 53.68 | 47.63 | 1
J211531.47+123957.5 | 592502156 | 318.881150 | 12.665982 | 52.55 | 35.69 | 17.17 | 39.60 | 35.41 | 1
J155144.87+002948.8 | 133709100 | 237.936982 | 0.496901 | 18.40 | 7.45 | 6.73 | 18.98 | 15.05 | 1
J071856.25+102638.3 | 446310014 | 109.734390 | 10.443990 | 64.56 | 35.33 | 24.47 | 53.17 | 49.80 | 1
J212356.68+153323.5 | 592415028 | 320.986190 | 15.556550 | 36.19 | 18.30 | 12.39 | 31.63 | 28.95 | 1
J210132.70+135622.5 | 371602229 | 315.386270 | 13.939585 | 37.99 | 29.44 | 10.61 | 27.60 | 22.89 | 2
J034322.46+463222.2 | 399504032 | 55.843622 | 46.539507 | 18.85 | 8.98 | 9.62 | 18.38 | 18.24 | 2
J045031.77+230712.9 | 197203189 | 72.632382 | 23.120270 | 41.74 | 26.74 | 23.16 | 33.79 | 40.42 | 2
J060619.60+200141.7 | 504103064 | 91.581701 | 20.028262 | 43.07 | 16.75 | 36.92 | 40.71 | 50.95 | 2
J182942.24+110429.1 | 746715115 | 277.426030 | 11.074758 | 38.31 | 16.52 | 17.89 | 29.15 | 31.51 | 2
J072222.78+445904.4 | 504804107 | 110.594950 | 44.984569 | 26.37 | 13.70 | 7.60 | 21.90 | 16.67 | 3
J122420.49+264738.8 | 714311122 | 186.085402 | 26.794137 | 17.85 | 8.02 | 7.97 | 13.39 | 13.35 | 3
J113143.40+370128.0 | 450104014 | 172.930840 | 37.024465 | 37.58 | 17.88 | 17.81 | 37.05 | 33.04 | 3
J101134.32+342150.7 | 732201084 | 152.893033 | 34.364110 | 2.92 | 0.85 | 3.17 | 4.18 | 4.56 | 3
J104052.58+284856.7 | 630714012 | 160.219100 | 28.815752 | 21.98 | 7.24 | 7.29 | 14.95 | 13.58 | 3
J222527.75+012522.5 | 484002197 | 336.365666 | 1.422931 | 5.74 | 3.83 | 2.23 | 4.17 | 4.64 | 4
J004511.44+321708.6 | 195305132 | 11.297683 | 32.285725 | 9.69 | 2.65 | 5.83 | 8.55 | 10.90 | 4
J004938.10+460953.1 | 498808130 | 12.408772 | 46.164764 | 5.17 | 1.94 | 6.25 | 6.29 | 9.29 | 4
J095728.68+275506.2 | 50401068 | 149.369516 | 27.918390 | 9.54 | 14.27 | 1.76 | 1.56 | 2.63 | 4
J105916.42+512443.1 | 148611045 | 164.818450 | 51.411984 | 7.49 | 3.89 | 1.52 | 6.28 | 4.22 | 4
… | … | … | … | … | … | … | … | … | …

Note. — Column 1-9: Basic information in the LAMOST catalog. Column 10: Type
description. 1: Hot subdwarfs that have been identified by previous work. 2:
Newly discovered Hot subdwarfs. 3: Identified as other classes by manual
validation. 4: Cannot determine the class due to low SNR.

Table 5: The atmospheric parameters of 25 newly identified Hot subdwarfs in this work. Designation† | RA∘‡ | Dec∘ | $T_{eff}$ |  $log$(g) |  $log$(nHe$/$nH)§ | SNR | uSDSS | gSDSS | G GaiaDR2
---|---|---|---|---|---|---|---|---|---
LAMOST | LAMOST | LAMOST | (K) | (cm$/$s-2) |  |  $\mu$-band | (mag) | (mag) | (mag)
J011346.77+002828.6 | 18.444903 | 0.474638 | 66154$\pm$377 | 6.5$\pm$0.03 | 2.38$\pm$0.45 | 63.05 | 14.51 | 14.96 | 15.23
J021334.02+435651.7 | 33.391767 | 43.947714 | 50287$\pm$1961 | 5.86$\pm$0.31 | -0.07$\pm$0.19 | 2.0 |  $\cdots$ |  $\cdots$ | 16.89
J023007.26+292256.2 | 37.530274 | 29.382299 | 74994$\pm$1650 | 6.50$\pm$0.08 | 1.89$\pm$2.49 | 21.37 | 16.99 | 17.39 | 17.61
J032425.51+430302.7 | 51.106302 | 43.050760 | 49080$\pm$511 | 6.19$\pm$0.08 | 0.76$\pm$0.13 | 6.28 | 17.52 | 17.58 | 17.67
J034322.46+463222.2 | 55.843622 | 46.539507 | 48674$\pm$346 | 5.81$\pm$0.07 | 1.00$\pm$0.12 | 8.98 | 16.80 | 16.73 | 16.70
J042351.08+322201.0 | 65.962855 | 32.366951 | 50248$\pm$977 | 5.73$\pm$0.13 | 0.11$\pm$0.13 | 3.61 | 17.38 | 17.39 | 17.45
J045031.77+230712.9 | 72.632382 | 23.120270 | 41776$\pm$75 | 5.79$\pm$0.02 | 0.84$\pm$0.02 | 26.74 | 17.73 | 17.56 | 17.47
J051135.26+391706.5 | 77.896933 | 39.285150 | 42434$\pm$273 | 6.31$\pm$0.09 | 1.08$\pm$0.20 | 4.47 |  $\cdots$ |  $\cdots$ | 17.13
J051304.75+114717.1 | 78.269827 | 11.788084 | 49634$\pm$257 | 6.50$\pm$0.01 | 2.07$\pm$0.11 | 73.99 |  $\cdots$ |  $\cdots$ | 14.38
J054559.57+222738.0 | 86.498229 | 22.460566 | 46032$\pm$797 | 5.52$\pm$0.11 | 1.03$\pm$0.24 | 8.75 |  $\cdots$ |  $\cdots$ | 17.63
J060619.60+200141.7 | 91.581701 | 20.028262 | 47105$\pm$321 | 5.87$\pm$0.07 | -0.25$\pm$0.03 | 16.75 |  $\cdots$ |  $\cdots$ | 16.90
J064529.73+412642.0 | 101.373890 | 41.445024 | 55297$\pm$0 | 5.07$\pm$0 | 2.50$\pm$0 | 4.81 |  $\cdots$ |  $\cdots$ | 16.61
J064558.40+111223.6 | 101.493340 | 11.206556 | 39831$\pm$329 | 5.54$\pm$0.06 | 0.74$\pm$0.08 | 4.66 | 17.44 | 17.52 | 17.68
J070700.65+193526.9 | 106.752730 | 19.590809 | 51530$\pm$469 | 6.50$\pm$0.07 | 1.08$\pm$0.20 | 12.07 |  $\cdots$ |  $\cdots$ | 17.42
J084223.13+375900.2 | 130.596410 | 37.983394 | 59127$\pm$263 | 6.27$\pm$0.10 | 2.40$\pm$3.53 | 12.98 | 17.04 | 17.51 | 17.86
J084350.85+361419.5 | 130.961886 | 36.238770 | 38293$\pm$166 | 5.62$\pm$0.01 | 2.27$\pm$0.04 | 7.12 | 16.87 | 17.01 | 17.23
J091029.42+090205.1 | 137.622608 | 9.034770 | 39329$\pm$235 | 6.23$\pm$0.02 | 1.50$\pm$0.04 | 5.05 | 17.00 | 17.09 | 17.26
J101457.79+451906.1 | 153.740813 | 45.318381 | 42182$\pm$172 | 5.61$\pm$0.01 | 0.57$\pm$0.05 | 9.99 | 19.64 | 17.69 | 19.01
J124931.17+750727.2 | 192.379883 | 75.124244 | 55650$\pm$2437 | 5.17$\pm$0.15 | -3.68$\pm$0.73 | 6.04 |  $\cdots$ |  $\cdots$ | 16.83
J180237.05+032550.1 | 270.654400 | 3.430598 | 44673$\pm$561 | 6.07$\pm$0.13 | 1.05$\pm$0.23 | 6.89 | 17.72 | 25.11 | 17.65
J181325.00+065315.9 | 273.354170 | 6.887751 | 50584$\pm$434 | 5.80$\pm$0.06 | 0.60$\pm$0.07 | 21.92 |  $\cdots$ |  $\cdots$ | 15.96
J182942.24+110429.1 | 277.426030 | 11.074758 | 52791$\pm$533 | 6.07$\pm$0.07 | 0.60$\pm$0.01 | 16.52 |  $\cdots$ |  $\cdots$ | 16.70
J205901.96+334933.0 | 314.758180 | 33.825842 | 74995$\pm$7667 | 5.97$\pm$0.27 | 2.14$\pm$11.20 | 4.01 |  $\cdots$ |  $\cdots$ | 17.49
J210132.70+135622.5 | 315.386270 | 13.939585 | 74981$\pm$251 | 6.50$\pm$0.01 | -0.33$\pm$0.06 | 29.44 |  $\cdots$ |  $\cdots$ | 16.49
J230829.86+214333.8 | 347.124442 | 21.726063 | 28470$\pm$536 | 5.07$\pm$0.11 | -2.86$\pm$0.13 | 3.23 | 17.16 | 15.94 | 15.60


# Detecting Adversarial Examples in Batches - a geometrical approach

Danush Kumar Venkatesh  Peter Steinbach

###### Abstract

Many deep learning methods have successfully solved complex tasks in computer
vision and speech recognition applications. Nonetheless, the robustness of
these models has been found to be vulnerable to perturbed inputs or
adversarial examples, which are imperceptible to the human eye, but lead the
model to erroneous output decisions. In this study, we adapt and introduce two
geometric metrics, _density_ and _coverage_ , and evaluate their use in
detecting adversarial samples in batches of unseen data. We empirically study
the metrics using MNIST and two real-world biomedical datasets from MedMNIST,
subjected to two different adversarial attacks. Our experiments show promising
results for both metrics to detect adversarial examples. We believe that his
work can lay the ground for further study on these metrics’ use in deployed
machine learning systems to monitor for possible attacks by adversarial
examples or related pathologies such as dataset shift.

Machine Learning, ICML

##  1 Introduction

Neural networks (NN) are machine learning algorithms that have been
demonstrated to achieve human-level performance in computer vision (e.g. deep
neural network architectures, DNNs), in speech recognition (Hinton et al.,
2012) and many other tasks of academic, industrial or cultural value. However,
studies have shown that the stability (or robustness) of such models can be
distorted by adding small human imperceptible perturbations to the input
samples (Szegedy et al., 2014; Goodfellow et al., 2014) so that predictions by
these systems are misguided. Such imperceptibly perturbed samples are termed
as _adversarial samples_ (AEs) and can be crafted by _adversarial attacks_
(Brendel et al., 2018). This unenviable property of the NNs causes major
security concerns with regard to deployment in real-world applications like
medical imaging (Lundervold & Lundervold, 2019) or self-driving cars (Evtimov
et al., 2017) to name a few.

#### Our Contributions

In this paper, we propose the use of _density_ and _coverage_ (Naeem et al.,
2020) as metrics for detecting AEs in batches of images. We have adopted these
two metrics from the field of generative modeling and quality assessment of
generative adversarial networks (GANs).

In detail, we contribute the following aspects to the literature:

  * •

We adopted _density_ and _coverage_ as metrics and propose a _model-agnostic_
method to detect adversarial samples in batches of images.

  * •

We empirically demonstrate the capacity of the metric using standard and real-
world datasets subjected to both white- and black-box attacks aligned to
recommendations in (Carlini & Wagner, 2017).

  * •

We study the root cause of _density_ and _coverage_ curves obtained from the
aforementioned experiments.

  * •

We have improved upon the computational efficiency of the proposed metrics.

Our work aspires to remain computational efficient and useful for
practitioners. In other words, our approach only requires practical
applications to ”train” a k-nearest neighbor tree on the used training set of
images or affiliated NN embeddings obtained from domain-specific application,
e.g. NN derived image classification. Due to the availability of performant
open-source libraries such as faiss (Johnson et al., 2017) we consider this to
be a minimal burden for practical concerns.

##  2 Related Work

In this section, we provide a short summary of AE detection techniques and
provide some context to image quality metrics from a generative modelling
perspective in order to provide context for _density_ and _coverage_.

#### Detecting Adversarial Examples

With a statistical approach, (Grosse et al., 2017) proposed a model agnostic
and kernel-based two-sample test. The mean discrepancy distance metric
(Gretton et al., 2012) was used to detect AEs from clean samples. Detectors
based on kernel density and Bayesian uncertainty estimates were presented to
distinguish AEs from noisy and clean samples (Feinman et al., 2017). A
manifold learning strategy based on expansion models was used to study the
local dimensionality of adversarial regions. Local Intrinsic Dimensionality
(LID) estimates were used further to distinguish AEs (Ma et al., 2018). Based
on the assumption that the feature space of models can be fitted to class
conditional Gaussian distributions by Gaussian discriminant analysis, a
confidence score was suggested to detect out-of-distribution and AEs based on
Mahalanobis distance (Lee et al., 2018). A k-nearest neighbor(k-NN) model was
fit to the feature space of DNNs utilizing influence function score (Cohen et
al., 2020). Similarly, neighbor context encoder (NCE), a transformer-based
detector, was proposed by (Mao et al., 2020). (Li & Li, 2017) built a cascade
classifier to distinguish AEs using the PCA of samples from each output layer
of DNNs. For a more detailed review of the field, we refer to (Aldahdooh et
al., 2022).

#### GAN image generation quality metrics

The qualitative performance of a GAN model is commonly assessed with respect
to two main aspects: fidelity (quality) and diversity (variability) of the
generated (fake) images with respect to the training data set. Multiple
metrics have been presented to evaluate the models based on these aspects.
More recently, two separate metrics, precision and recall, were formulated
based on the uniform density assumptions of real and fake data distributions
(Sajjadi et al., 2018). Further (Kynkäänniemi et al., 2019) proposed improved
precision (IP) and recall (IR), wherein the probability density functions were
used to cautiously construct the manifolds using the _k_ th nearest neighbor.
Certain drawbacks of IP&IR such as outlier susceptibility or the computational
expense were overcome by _density_ and _coverage_ , that was implemented as
prdc (Naeem et al., 2020).

##  3 Adversarial example detection

In this work, we focus on image classification and attacking such a trained
model to produce AEs in an untargeted fashion, i.e., the model predicts any
other class than the ground truth. In this section, we introduce _density_ and
_coverage_ mathematically.

Let $\mathcal{P}$(X) be the distribution of the dataset images and
$\mathcal{Y}$ be the corresponding class labels. Following the underlying
assumptions from prdc, we sample two different distributions, the model
dataset $\mathcal{P}(X_{m})$ and a hold-out set, the validation dataset
$\mathcal{P}(X_{v})$. Let the labeled set, $S_{m}=(x_{i},y_{i})^{m}_{i=1}$ be
sampled from $\mathcal{P}(X_{m})$ and $S_{v}=(x_{j},y_{j})^{n}_{j=1}$ from
$\mathcal{P}(X_{v})$, with _m_ and _n_ the number of samples in model and
validation dataset respectively. Let $f$ be a model trained on
$\mathcal{P}(X_{m})$. The model $f$ is then attacked using $S_{v}$ to generate
the adversarial sample $x_{a}$ with the distribution $\mathcal{P}(X_{a})$.

###  3.1 Density

In the context of generative modeling, density quantifies the quality and
coverage assesses the variability of the generated (fake) images. These
metrics, have been developed with improved precision and recall(IP&IR) as
their backbone. Let $\mathcal{R}$ and $\mathcal{Q}$ be reference and query
distributions, then IP&IR is computed by constructing the manifolds $\Phi_{r}$
and $\Phi_{q}$, one for each distribution, with the hypersphere radii
estimated by the _k_ -nearest neighbor. The binary decision of whether $q_{i}$
sampled from query distribution $\mathcal{Q}$ lies in the manifold $\Phi_{r}$
constitutes precision, whereas recall measures the portion of $\mathcal{R}$
falling inside $\Phi_{q}$. _Density_ is computed by normalizing the
constructed manifold and counting the number of reference manifold spheres
containing $q_{i}$. It is defined as

$\text{density}=\frac{1}{kS}\sum_{j=1}^{S}\sum_{i=1}^{T}1_{q_{j}\in\mathcal{M}(r_{1},\cdots,r_{S})}$
(1)

where _S_ and _T_ are the number of reference and query samples, $1_{(.)}$ the
binary decision function and $\mathcal{M}$ the defined reference manifold.

###  3.2 Coverage

_Coverage_ computation focuses on constructing manifolds only for the
reference samples in contrast to IR and measures the fraction of these
manifolds containing at least one query sample. _Coverage_ is defined as

$\text{coverage}=\frac{1}{T}\sum_{i=1}^{T}1_{\exists j\text{ s.t.
}q_{j}\in\mathcal{M}(r_{1},\cdots,r_{S})}$ (2)

_Density_ remains unbounded, that is, it can be larger than $1$, whereas
_coverage_ is bounded between $0$ and $1$. The manifolds are defined by

$\mathcal{M}\left(r_{1},\cdots,r_{S}\right)=\bigcup_{i=1}^{S}\mathcal{H}\left(r_{i},\operatorname{nn}_{k}\left(r_{i}\right)\right)$
(3)

where $\mathcal{H}(r_{i},\tilde{r})$ is the sphere around $r_{i}$, with the
radius $\tilde{r}$ estimated by $\operatorname{nn}_{k}\left(r_{i}\right)$ that
denotes the _k_ nearest neighbor to $r_{i}$, expressed in $\mathbf{R}^{d}$.
The computation of these manifolds exposes a severe runtime overhead in the
original implementation of prdc. We overcome this problem using the tree
structure of faiss (Johnson et al., 2017) to find the nearest neighbor to a
given data point and efficiently construct the hyperspheres. Appendix D is
referred for related analysis.

##  4 Experiments

We briefly explain the datasets and the adversarial attacks which were
explored to quantify the proposed metric.

###  4.1 Datasets

The study was performed on the standard MNIST dataset (LeCun et al., 1998) and
also on 2D real-world biomedical datasets from MedMNIST (Yang et al., 2021).
The datasets are described as follows,

MNIST: The dataset consists of $70000$ grayscale images with $10$ classes
corresponding to handwritten digits $0$ to $9$. The dataset was split into a
model dataset size of $67900$ images and a validation dataset of $2100$
images.

PathMNIST: This dataset consists of $107,180$ pathological tissue images
consisting of $9$ classes describing different pathological tissue types. A
total of $5359$ images were used to form the validation dataset, and the
remaining was the model dataset. This dataset is encoded as a set of RGB
images (Kather et al., 2019).

OrganMNIST: $58,850$ abdominal CT scan grayscale images corresponding to $11$
different classes are contained in this dataset (Bilic et al., 2019). A
validation set size of $4708$ images was chosen for our study.

All the images were of $28$x$28$ pixels. Each dataset was split using a
stratification strategy. The proportional values are available in Appendix B.
For illustration, we show a subset of each dataset in Figure 1.

---

(a) MNIST

(b) PathMNIST

(c) OrganMNIST

Figure 1: The images of the different dataset. Each image represents a unique
class in the dataset.

###  4.2 Procedure for metric computation

A necessary preliminary step for the computation of the metric and the
interpretation of our results is the dataset curation. We conducted this as
follows:

  1. 1.

Dataset split: The dataset of interest is split into $2$ different sets, a set
to train the model, and a hold-out dataset. We mention these splits as the
model dataset and the validation dataset. The model dataset is used to train
the CNN model, and the validation dataset is used exclusively to perform the
attack and generate adversarial samples.

  2. 2.

Generate adversarial samples: Attack the trained model using the validation
dataset and generate the _validation adversarial samples_

However, if adversarial samples are already present, step 2 can be bypassed.

The procedure to differentiate and detect the adversarial samples using the
proposed metrics is as follows:

  1. 1.

Reference _density_ /_coverage_ : compute the metrics between the _model
dataset_ and the _validation dataset_ which serves as reference value before
the model is attacked.

  2. 2.

Adversarial _density_ /_coverage_ : compute the metrics between the _model
dataset_ and the _validation adversarial samples_.

The reference manifold is always constructed on the model dataset($\Phi_{m}$)
and the query samples constitute either the validation dataset or the
generated AEs to compute the reference and adversarial metric respectively.

In our setup, _density_ and _coverage_ are metrics to describe the geometric
distribution of samples from the holdout set (e.g. a query batch) with respect
to the distribution of the data used for training. Along this line of thought,
our method can be applied on any input data that represents a query batch
uniquely in an appropriate n-dimensional space. In other words, these metrics
can be computed on the entire benign or malignant query image batch.
Alternatively, both metrics can also be calculated from the embedded space of
logits of a trained classifier or autoencoder as obtained from the query
image. In the following, we will report the results obtained for the entire
images only. Interested readers curious about the outcomes in the embedded
space are kindly deferred to Appendix C.

###  4.3 Adversarial attacks

We focus our study on two attacks namely, Fast gradient sign method (FGSM)
(Goodfellow et al., 2014) and the Boundary attack (Brendel et al., 2018). We
use these two extremes to probe the rich space of possible attacks already
published. As our method is independent of the attack, we consider it highly
likely that our reported results map to other attacks too.

#### FGSM

A type of white-box attack in which AEs are crafted by computing the gradient
of the loss function for the given input image such that the loss increases
along the steepest direction. In our study, it was meaningful to vary the
magnitude of perturbation, $\epsilon$, in the range $[0,1]$ in steps of $0.05$
as the accuracy of the models decreased to below-par values.

#### Boundary attack

In this algorithm, AEs are generated by reducing the distance between the
original (non-adversarial) sample and adversarial target (i.e., a sample from
a target class to be injected) by tracing along the decision boundary and
making a step-update in combination with a suitable perturbation from a given
distribution.

Foolbox (Rauber et al., 2017) was used to attack our trained models and to
generate the AEs.

###  4.4 CNN Model

We designed two different model architectures for the classifier model on
which the attack was performed: A $5$ layer model for the MNIST dataset. For
both the MedMNIST datasets, an architecture of $5$ convolutional layers
followed by $3$ fully-connected layers was created. Both MedMNIST models are
only different in the number of channels for the input layer. The model
details are provided in Appendix B.

To define the uncertainties of the metrics for the attacks, uncertainty
estimates were obtained for both reference and adversarially altered metrics.
For this, the respective image datasets were split into smaller batches of
$100$ images each. Then, any metric of interest was collected for each of
these sub-batches. From the ensemble of measurements obtained this way, we
compute the $0.25$, $0.5$, and $0.75$ quantiles of any metrics of interest for
this study.

For the FGSM attack, we use $20$ values for epsilon, and both metrics are
computed for each of these values. In the case of the Boundary attack, we also
examine the metrics by varying the batch size of the images depending on the
datasets. Throughout this study, $k$ as of Equation 1 and Equation 2 was
maintained at a value of $5$ in any nearest neighbor query performed. The code
and the implementation of experiments are provided in (anonymized authors, ).

##  5 Results & Discussion

We summarize and discuss the results of the experiments in this section. The
CNN model trained for the MNIST dataset achieved an accuracy of $99.2~{}\%$.
Similarly, the models for the medical images were trained to achieve an
accuracy of $96~{}\%$ and $99.1~{}\%$ for PathMNIST and OrganMNIST,
respectively. The model accuracy for biomedical images was on par with the
benchmark results from (Yang et al., 2021). The AEs generated for different
datasets are shown in Figure 2 to provide an overview and intuition.

###  5.1 FGSM attack

The variation of _density_ and _coverage_ for the FGSM attack on the datasets
is shown in Figure 3(a).

Dataset |  Original | FGSM |  Bdy.
---|---|---|---
MNIST |  |  |  |  |
|  |  |  |  |
PathMNIST |  |  |  |  |
|  |  |  |  |
OrganMNIST |  |  |  |  |
|  |  |  |  |
$\epsilon_{\text{FGSM}}$ |  |  $0.3$ |  $0.75$ |  $1.0$ |
Figure 2: Example images from the original datasets as well as generated by
two adversarial attacks from our classification models. The $2^{nd}$ column
indicates the original (benign) samples. $3^{rd}$, $4^{th}$ and $5^{th}$
columns correspond to FGSM attack with $\epsilon_{\text{FGSM}}$ being set to
$0.3,0.75$ and $1$ as indicated. The last column of images are AEs generated
from the boundary attack.

#### MNIST

The reference _density_ for the MNIST dataset achieves a value of $0.997$
(horizontal line in Figure 3(a) $1^{st}$ row). After the attack, the
adversarial _density_ varies from the reference _density_ for the complete
range of epsilon values. It increases gradually with an increase in epsilon
and crosses over towards small values of _density_ at $\epsilon=0.38$, after
which it gradually decreases further with the increase in epsilon. The
reference _coverage_ for the dataset has a value of $0.39$. Similar to
adversarial _density_ , the adversarial _coverage_ for the MNIST dataset
increases initially with a peak value of $0.46$, then crosses over the
reference value at $\epsilon=0.28$ and decreases further towards smaller
values.

#### MedMNIST

The _density_ variation for the OrganMNIST dataset shows similar behavior to
the MNIST dataset. The reference _density_ has a value of $1.01$, and the
adversarial _density_ peaks at $1.17$ and reduces to $0.22$ for the maximum
epsilon value. However, for the case of PathMNIST, a non-identical trend in
the adversarial _density_ was noticed. This dataset has a reference _density_
value of $0.83$, and the adversarial _density_ raises thereafter with the
increase in epsilon, having the maximum value of $4.56$. The behavior and
variation in _coverage_ for all three datasets appears similar to each other.
In the case of OrganMNIST, the reference _coverage_ had a value of $0.34$, and
the adversarial _coverage_ varies between $0.36$ and $0.003$. The adversarial
_coverage_ has a sharp peak at $\epsilon=0.18$ for PathMNIST dataset, whereas
the reference _coverage_ remained at $0.49$.

#### Metric performance

As described for each dataset, the reference _density_ lies close to the value
of $1$. This is an expected outcome, as it confirms the i.i.d assumption. It
also indicates that the model dataset manifold $\Phi_{m}$ is closely packed
without outliers along with the validation samples lying in the vicinity
(Naeem et al., 2020). From the Figure 3(a), it is evident that for the full
range of epsilon values, both adversarial _density_ and _coverage_ exhibit a
noticeable difference from the reference metric. For the MNIST and OrganMNIST
datasets, the adversarial _density_ increases in the region between
$\epsilon=0$ and $\epsilon=0.38$, which is purely due to geometric reasons.
Concerning the definition of the FGSM attack, the gradient tensor with respect
to the input image is nudged in a direction with the scaling value of epsilon
to create an adversarial sample. As the computation of _density_ and
_coverage_ is based on the binary function, as seen in Equation 1 and Equation
3, with the initial increase of epsilon, the generated AEs move into the
multitude of model dataset manifolds $\Phi_{m}$, and this augments the number
of hyperspheres $\mathcal{H}$ containing AEs thereby contributing to the
initial increase of the metric.

With higher epsilon values, the decreasing trend of the _density_ curves
underlines that the manifold, $\Phi_{m}$ and AEs become separated. The
continuous increase of adversarial _density_ in the case of PathMNIST revealed
that the generated AEs are classified into a particular class with the
increase in epsilon, thereby confining the AEs into a small region of
hyperspheres and contributing to the _density_ increase. As the value obtained
for _coverage_ depends on the number of samples considered for its
computation, a variation in reference values _coverage_ between the datasets
is expected. From the _coverage_ plots, the decreasing _coverage_ values with
the increase in epsilon indicate the reduction in the diversity of samples
generated. That is, the generated AEs always tend to be classified into a
particular class with the increase of epsilon. A visualization in a space of
reduced dimensionality using pymde (Agrawal et al., 2021) and an analysis of
it are provided in Appendix C.

(a) FGSM attack

(b) Boundary attack

Figure 3: The variation of _density_ and _coverage_ for the reference
validation dataset as well as for AEs obtained from it. The red line
corresponds to metrics obtained from the benign (reference) validation set.
The blue line refers metrics obtained from malignant AEs derived from the same
validation set. The green and gray bands indicate uncertainty intervals
between $25\%-50\%$ and $50\%-75\%$ quantiles estimated from ensemble
measurements respectively. Each column corresponds to the specified dataset as
indicated. Rows report results for _density_ or _coverage_ respectively.
_Density_ and _coverage_ from AEs are distinctly different from reference
values.

###  5.2 Boundary attack

The behaviour of _density_ and _coverage_ between the benign and AEs generated
using the boundary attack for the datasets are shown in Figure 3(b).

#### MNIST

For this dataset, the reference _density_ lies between $0.9$ and $1.1$ for
various batch sizes. The adversarial _density_ values were comparatively low,
within the range of $0.11$ and $0.22$. The reference _coverage_ achieves a
value of $0.14$ compared to the adversarial _coverage_ value of $0.004$ for
the entire batch of our validation dataset. An interesting result, as noticed
in the $2^{nd}$ column of Figure 3(b), is that _coverage_ increases gradually
with an increase in image batch sizes and finally reaches that of reference or
adversarial values.

#### MedMNIST

In the case of PathMNIST, an averse behavior of _density_ to that of MNIST was
noticed. The reference _density_ has a value of $0.831$ and the adversarial
_density_ with the value of $10.12$ for the whole batch of validation dataset.
A slight difference in values was measured in the OrganMNIST dataset,:with
reference _density_ lying between $0.96$ and $1.05$ compared to adversarial
_density_ values of $2.2$ and $3.1$. The _coverage_ values for these datasets
showed the same inclination as seen in the MNIST dataset: the gradual increase
of _coverage_ with the increase in batch sizes. For the entire batch of
images, we obtained the reference _coverage_ for PathMNIST at a value of
$0.49$ and the adversarial _coverage_ value of $0.65$. For the OrganMNIST, the
reference _coverage_ was at $0.34$ in meager contrast to adversarial
_coverage_ of $0.24$.

#### Metric performance

Similar to the FGSM attack, the reference values of density and coverage
remained the same. However, for the case of MNIST and PathMNIST datasets, a
discernible difference was noticed between the reference and adversarial
metrics. The progressive increase of _coverage_ with an increase in the batch
size reassures us of the dependency on the number of samples involved in its
computation. The results also exhibit the distinctive capability of _density_
and _coverage_ to detect the adversarial samples. The variation in these
metrics reveals that the pathological characteristics of AEs were more adverse
than those crafted by the FGSM attack.

###  5.3 Admixture of adversarial samples

In a real-world scenario, practitioners aspire to detect AEs in batches of
samples to classify on a case by case basis at best. In order to learn, how
precise _density_ and _coverage_ can be used for this purpose we intermixed
AEs into subsets of benign (untouched) samples in proportionate quantities. We
then computed our metrics of interest. The metrics variation for each of the
dataset for the FGSM attack is shown in Figure 4. The results of the similar
analysis for the boundary attack is available in the supplementary material.

As seen from these results, _density_ and _coverage_ show a distinctively
progressive deviation from the full batch of benign samples (i.e. a deviation
away from the reference metric), to that of the batch of AEs for the attack.
These experiments further advocate that _density_ and _coverage_ can serve as
a measure to detect AEs.

Figure 4: The proportionate mixture of adversarial samples and benign samples
for the FGSM attack. _Density_ and _coverage_ for the entire batch of
benign(reference) and adversarial samples are indicated by black and purple
lines, respectively. The red, blue, and green lines indicate the variation of
the metrics for $25\%$, $50\%$, and $75\%$ proportional mixture of AEs with
benign samples. _Density_ variation is specified in the first row, followed by
_coverage_ in the second row. Above values of 25% of malignant sample
admixture, _density_ and _coverage_ values start to diverge above dataset-
specific thresholds.

###  5.4 Discussion

An essential property of our proposed metrics is that they are model-agnostic.
In that spirit, they can directly operate on the feature space or the data
itself. This overlooks the development of auxillary models for AE detection.
In contrast to the usefulness of _density_ and _coverage_ for detection of
AEs, we see a few shortcomings from our analysis above: for one, the
sensitivity of _coverage_ to the number of samples; further the number of
nearest neighbors used for computation, which is inherent to the mathematical
formulation of _coverage_ computation is another.

##  6 Conclusion

We have adopted two metrics for quality of sample generation from generative
modeling, i.e. _density_ and _coverage_. We improved their design to make them
of practical use in realistic runtime scenarios. We then studied how _density_
and _coverage_ behave in the face of AEs for classifying images from MNIST or
from benchmark datasets in the medical domain, PathMNIST and OrganMNIST. Our
experiments confirm that both _density_ and _coverage_ are susceptible to
effects maliciously introduced in AEs which are imperceptible to humans.
Moreover, we demonstrated that admixtures of AEs into batches of untouched
benign images by $25\%$ or more distort the values of _density_ or _coverage_
in such a fashion, that it makes both metrics viable candidates to flag such
batches of images as malignant in practice. We believe that this can and
should be subject of further study. With a potential addition of statistical
hypothesis tests to differentiate provided (unseen) samples from a known
training set using _density_ and _coverage_ , our approach can be of
substantial support in fraud detection and monitoring of adversarial attacks
in deployed machine learning systems. Apart from this, applications in
detecting dataset shift or similar pathologies specific to a presented dataset
in comparison to the training set should also be considered in future work.

## References

  * Agrawal et al. (2021) Agrawal, A., Ali, A., and Boyd, S.  Minimum-distortion embedding.  _arXiv_ , 2021.
  * Aldahdooh et al. (2022) Aldahdooh, A., Hamidouche, W., Fezza, S. A., and Déforges, O.  Adversarial example detection for dnn models: A review and experimental comparison.  _Artificial Intelligence Review_ , pp. 1–60, 2022.
  * (3) anonymized authors, V.  n2gem.  URL https://anonymous.4open.science/r/n2gem/README.md.  anonymized on Mar 21, 2022.
  * Beyer et al. (2020) Beyer, L., Hénaff, O. J., Kolesnikov, A., Zhai, X., and Oord, A. v. d.  Are we done with imagenet?, 2020.  URL https://arxiv.org/abs/2006.07159.
  * Biggio & Roli (2018) Biggio, B. and Roli, F.  Wild patterns: Ten years after the rise of adversarial machine learning.  _Pattern Recognition_ , 84:317–331, 2018.  ISSN 0031-3203.  doi: https://doi.org/10.1016/j.patcog.2018.07.023.
  * Bilic et al. (2019) Bilic, P., Christ, P. F., Vorontsov, E., Chlebus, G., Chen, H., Dou, Q., Fu, C.-W., Han, X., Heng, P.-A., Hesser, J., et al.  The liver tumor segmentation benchmark (lits).  _arXiv preprint arXiv:1901.04056_ , 2019.
  * Brendel et al. (2018) Brendel, W., Rauber, J., and Bethge, M.  Decision-based adversarial attacks: Reliable attacks against black-box machine learning models.  In _International Conference on Learning Representations_ , 2018.
  * Carlini & Wagner (2017) Carlini, N. and Wagner, D.  Adversarial examples are not easily detected: Bypassing ten detection methods.  In _Proceedings of the 10th ACM workshop on artificial intelligence and security_ , pp. 3–14, 2017.
  * Carter et al. (2021) Carter, B., Jain, S., Mueller, J. W., and Gifford, D.  Overinterpretation reveals image classification model pathologies.  _Advances in Neural Information Processing Systems_ , 34, 2021.
  * Cohen et al. (2020) Cohen, G., Sapiro, G., and Giryes, R.  Detecting adversarial samples using influence functions and nearest neighbors.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 14453–14462, 2020.
  * Evtimov et al. (2017) Evtimov, I., Eykholt, K., Fernandes, E., Kohno, T., Li, B., Prakash, A., Rahmati, A., and Song, D.  Robust physical-world attacks on machine learning models.  _arXiv preprint arXiv:1707.08945_ , 2(3):4, 2017\.
  * Feinman et al. (2017) Feinman, R., Curtin, R. R., Shintre, S., and Gardner, A. B.  Detecting adversarial samples from artifacts.  _arXiv preprint arXiv:1703.00410_ , 2017.
  * Finlayson et al. (2019) Finlayson, S. G., Bowers, J. D., Ito, J., Zittrain, J. L., Beam, A. L., and Kohane, I. S.  Adversarial attacks on medical machine learning.  _Science_ , 363(6433):1287–1289, 2019.
  * Goodfellow et al. (2014) Goodfellow, I. J., Shlens, J., and Szegedy, C.  Explaining and harnessing adversarial examples.  _arXiv preprint arXiv:1412.6572_ , 2014.
  * Gretton et al. (2012) Gretton, A., Borgwardt, K. M., Rasch, M. J., Schölkopf, B., and Smola, A.  A kernel two-sample test.  _The Journal of Machine Learning Research_ , 13(1):723–773, 2012.
  * Grosse et al. (2017) Grosse, K., Manoharan, P., Papernot, N., Backes, M., and McDaniel, P.  On the (statistical) detection of adversarial examples.  _arXiv preprint arXiv:1702.06280_ , 2017.
  * Hinton et al. (2012) Hinton, G., Deng, L., Yu, D., Dahl, G. E., Mohamed, A.-r., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T. N., et al.  Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups.  _IEEE Signal processing magazine_ , 29(6):82–97, 2012.
  * Howard & Gugger (2020) Howard, J. and Gugger, S.  _Deep Learning for Coders with fastai and PyTorch_.  O’Reilly Media, 2020.
  * Johnson et al. (2017) Johnson, J., Douze, M., and Jégou, H.  Billion-scale similarity search with gpus.  _arXiv preprint arXiv:1702.08734_ , 2017.
  * Kather et al. (2019) Kather, J. N., Krisam, J., Charoentong, P., Luedde, T., Herpel, E., Weis, C.-A., Gaiser, T., Marx, A., Valous, N. A., Ferber, D., et al.  Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study.  _PLoS medicine_ , 16(1):e1002730, 2019.
  * Kynkäänniemi et al. (2019) Kynkäänniemi, T., Karras, T., Laine, S., Lehtinen, J., and Aila, T.  Improved precision and recall metric for assessing generative models.  _Advances in Neural Information Processing Systems_ , 32:3927–3936, 2019.
  * LeCun et al. (1998) LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P.  Gradient-based learning applied to document recognition.  _Proceedings of the IEEE_ , 86(11):2278–2324, 1998.
  * Lee et al. (2018) Lee, K., Lee, K., Lee, H., and Shin, J.  A simple unified framework for detecting out-of-distribution samples and adversarial attacks.  _Advances in neural information processing systems_ , 31, 2018.
  * Li & Li (2017) Li, X. and Li, F.  Adversarial examples detection in deep networks with convolutional filter statistics.  In _Proceedings of the IEEE International Conference on Computer Vision_ , pp. 5764–5772, 2017.
  * Lundervold & Lundervold (2019) Lundervold, A. S. and Lundervold, A.  An overview of deep learning in medical imaging focusing on mri.  _Zeitschrift für Medizinische Physik_ , 29(2):102–127, 2019.
  * Ma et al. (2018) Ma, X., Li, B., Wang, Y., Erfani, S. M., Wijewickrema, S., Schoenebeck, G., Song, D., Houle, M. E., and Bailey, J.  Characterizing adversarial subspaces using local intrinsic dimensionality.  _arXiv preprint arXiv:1801.02613_ , 2018.
  * Mao et al. (2020) Mao, X., Chen, Y., Li, Y., He, Y., and Xue, H.  Learning to characterize adversarial subspaces.  In _ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 2438–2442. IEEE, 2020.
  * Naeem et al. (2020) Naeem, M. F., Oh, S. J., Uh, Y., Choi, Y., and Yoo, J.  Reliable fidelity and diversity metrics for generative models.  In _International Conference on Machine Learning_ , pp. 7176–7185. PMLR, 2020.
  * Rauber et al. (2017) Rauber, J., Brendel, W., and Bethge, M.  Foolbox: A python toolbox to benchmark the robustness of machine learning models.  In _Reliable Machine Learning in the Wild Workshop, 34th International Conference on Machine Learning_ , 2017.  URL http://arxiv.org/abs/1707.04131.
  * Sajjadi et al. (2018) Sajjadi, M. S., Bachem, O., Lucic, M., Bousquet, O., and Gelly, S.  Assessing generative models via precision and recall.  In _Proceedings of the 32nd International Conference on Neural Information Processing Systems_ , pp. 5234–5243, 2018.
  * Szegedy et al. (2014) Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R.  Intriguing properties of neural networks.  In _2nd International Conference on Learning Representations, ICLR 2014_ , 2014.
  * Yang et al. (2021) Yang, J., Shi, R., Wei, D., Liu, Z., Zhao, L., Ke, B., Pfister, H., and Ni, B.  Medmnist v2: A large-scale lightweight benchmark for 2d and 3d biomedical image classification.  _arXiv preprint arXiv:2110.14795_ , 2021.

##  Appendix A Adversarial attacks

We followed the categorization of adversarial attacks alongside (Biggio &
Roli, 2018) into:

  * •

White-box-attacks: The attacker has complete information regarding the target
model, including the model parameters and the architecture. AEs are crafted by
formulating an optimization problem.

  * •

Gray-box-attacks: In this category, the attacker has information regarding the
model architecture or the training data used to devise the parameters but no
direct access to the learned parameters.

  * •

Black-box-attacks: Adversarial examples are crafted by a query to the trained
model and observing the predicted label.

#### FGSM

The adversarial sample for the attack is formally defined as,

$x_{a}=x+\epsilon*sign(\nabla_{x}J(\theta,x,y))$ (4)

where $\epsilon$ is the magnitude of perturbation, $\theta$ are the network
parameters, and $J(\theta,x,y)$ is the cost or loss function of the trained
model.

#### Boundary attack

This is a rejection sampling algorithm to construct AEs. An adversarial target
(i.e. a sample from a target class to be injected) is used as the starting
point in this attack. A random perturbation drawn from the Gaussian
distribution is added to the benign source image iteratively. Then, a step-
update along the decision boundary between the original (non-adversarial)
sample and the AE is performed such that the distance between them is
minimized. Since this attack has information only about the predicted class
label, multiple queries to the model have to be made before convergence, i.e.
before a misclassification based on the AE is performed by the victim model.
The AEs are generated by the minimization $\lVert x-x_{a}\rVert_{2}^{2}$, such
that the new adversarial is defined by $x_{a}=x^{s-1}_{a}+\eta_{s}$, where
$\eta_{s}$ is the random perturbation from a suitable distribution for a
particular step $s$.

##  Appendix B Model information

The model architectures for the MNIST and the MedMNIST dataset are shown in
Table 1 and Table 2 respectively. Throughout the model, the kernel size was
maintained at with radius $3$ along with a drop-out rate of $0.5$. To have a
stabilizing effect on learning and to speed up the learning process, _batch
normalization_ was introduced in the convolution layers. Except for the last
layer, _ReLU_ was applied as the activation function and the _softmax
activation_ for the last layer. The Adam optimizer and negative log-likelihood
loss function was used. To estimate optimizer hyper parameters a $1$ cycle
policy was employed, wherein the learning rate is varied for each mini-batch
to train the model. The optimal learning rate for the dataset was found and
the model was fine-tuned to achieve the best possible outcome. All models were
trained using FastAI (Howard & Gugger, 2020). The split of samples into the
model dataset and validation set for the datasets are indicated in Table 3.

Layer No. |  Layer information | Description
---|---|---
1 |  Conv2D + ReLU + MaxPool | 16 filters
2 |  Conv2D + ReLU + MaxPool + Dropout | 32 filters
3 |  Flatten | 800 neurons
3 |  Dense + ReLU + Dropout | 512 neurons
4 |  Dense + ReLU | 64 neurons
5 |  Dense + Softmax | 10 classes
Table 1: The MNIST classifier architecture Layer No. |  Layer information | Description
---|---|---
1 |  Conv2D + Batchnorm + ReLU | 32 filters
2 |  Conv2D + Batchnorm + ReLU + MaxPool | 32 filters
3 |  Conv2D + Batchnorm + ReLU | 64 filters
4 |  Conv2D + Batchnorm + ReLU | 64 filters
4 |  Conv2D(padding=$1$) + Batchnorm + ReLU + MaxPool | 64 filters
5 |  Dense + ReLU | 128 neurons
6 |  Dense + ReLU | 64 neurons
7 |  Dense + Softmax | *classes
Table 2: The MedMNIST classifier architecture. *classes$=9$ and $11$ for PathMNIST and OrganMNIST respectively Dataset |  Total no. of samples |  No. of samples in model dataset |  No. of samples in validation set
---|---|---|---
MNIST |  $70000$ |  $67000$ |  $2100$
PathMNIST |  $107180$ |  $101821$ |  $5359$
OrgMNIST |  $58850$ |  $54142$ |  $4708$
Table 3: The number of samples considered in each of the datasets

##  Appendix C Metrics analysis

The value of density and coverage depends on the output of the binary
function—a short explanation of how density increases or decreases are shown
in Figure 5.

Figure 5: The graphical interpretation of the metric. The blue circles
indicate the manifold of the benign samples and the red points correspond to
AEs. The manifolds are formed with $k=2$.

The plot from left to right can be associated with the metrics variation for
initial values of epsilon. As epsilon increases for the FGSM attack, the AEs
(_red points_) move into the benign sample hyperspheres (_blue spheres_) and
augment the number of hyperspheres for the AE. This contributes to the
increase of the metric. However, with higher epsilon values, the AEs have
moved away from the benign sample manifolds, thereby decreasing the metrics.
For the PathMNIST dataset, an adverse behavior of adversarial density is
noticed. As epsilon increases, the AEs initially move into the benign
hyperspheres and get accommodated thereafter; the AEs for this dataset lies
close to the benign samples. This is reflected in the continuous increase of
density.

The difference in the model’s predictions to the true label for the range of
epsilon values for each dataset is shown in the histogram plots, i.e. Figure
6, Figure 7 and Figure 8. For the MNIST dataset, see Figure 6, beyond values
of $\epsilon=0.55$, class $8$ is predicted as the dominant class, which is
also evident from Figure 9(a) (increase in the number of _blue points_). We
also analyzed the relative change in volume of each cluster of the predicted
class, see Figure 10(a). This indicated that, the volume of the cluster
pertaining to class $8$ gradually increases with an increase in epsilon (the
yellow color indicates large difference in values). This can be mapped to
cluster $8$ attracting more and more malignant samples and eventually becoming
the dominant predicted class label. For the PathMNIST dataset, the class $2$
is the predominantly predicted class as seen from Figure 7 for high values of
$\epsilon$. As noticed from Figure 10(b) the volume of each class cluster
decreases and also remain nearly the same (the black regions for
$\epsilon=0.75,1$) with an increase in epsilon as compared to class $2$. The
Figure 9(b) further indicates that for higher values of epsilon, the classes
cluster into a confined space contributing to the increase of density well
above $1$. On the other hand, the coverage decreases as the diversity in the
generated AEs decreases. A similar trend to the MNIST dataset was also noticed
in the case of the OrganMNIST dataset.

We also analyzed the change in distance between each cluster center between
the benign samples and the generated AEs. The pair-wise distance between the
benign and AEs were computed, which then served as input for Multi-dimensional
scaling to obtain the reduced dimensional coordinates for the cluster centres.
As seen from Figure 11(a) and Figure 11(b), the distance matrix varies for
each epsilon value indicating an evident change in cluster centers for MNIST
and PathmMNIST datasets. However, the Figure 11(c) shows that such onset in
difference appears only after $\epsilon=0.5$ for the OrganMNIST dataset. These
results also purport the variation of the metrics explained in Figure 5.

The results of an analysis using the embeddings from the last layer of a
classification model in contrast to using the input images are shown in Figure
12. There exists a perceptible difference between the reference and
adversarial _density_ and _coverage_ for each of the datasets. We perceive
this as a clear indication, that the morphology of the embedded space is
severely different to that of the input space of raw images. However, we see
that AEs can be differentiated from benign samples in this scenarios too
albeit under different morphological constraints.

Figure 6: The variation between the true and predicted adversarial labels of
the MNIST dataset subjected to FGSM attack for different value of epsilons.
Figure 7: The variation for the PathMNIST dataset between the true and
predicted adversarial labels for different values of epsilons. This
illustrates how the attack changes the predicted class labels. Figure 8:
Variation between the true and predicted adversarial labels of the OrganMNIST
dataset subjected to FGSM attack for different values of epsilons. This
illustrates how the attack changes the predicted class labels.

(a) MNIST

(b) PathMNIST

(c) OrganMNIST

Figure 9: Reduced dimensionality representation of the adversarial samples for
each of the dataset. The colorbar indicates the label of each class in the
dataset.

(a) MNIST

(b) PathMNIST

(c) OrganMNIST

Figure 10: The difference in volume between the class clusters for each
epsilon. The plots are symmetric, with yellow indicating a distinct difference
between cluster volume and black no difference at all.

(a) MNIST

(b) PathMNIST

(c) OrganMNIST

Figure 11: The difference in cluster centres between the classes for each
epsilon. The y-axis indicates the class labels for benign samples and the
y-axis for AEs. The variation of the distance matrices between epsilons
indicate a change in cluster distance. Figure 12: The variation of _density_
and _coverage_ for the reference validation dataset as well as for AEs using
the embeddings. The red line corresponds to metrics obtained from the benign
(reference) validation set. The blue line refers metrics obtained from
malignant AEs derived from the same validation set. The green and gray bands
indicate uncertainty intervals between $25\%-50\%$ and $50\%-75\%$ quantiles
estimated from ensemble measurements respectively. Each column corresponds to
the specified dataset as indicated. Rows report results for _density_ or
_coverage_ respectively. _Density_ and _coverage_ from AEs are distinctly
different from reference values.

Similar to the FGSM attack, we performed the experiment of mixing AEs into the
full batch of validation(benign) samples for the boundary attack. The
variation in _density_ and _coverage_ values is shown in Table 4 and Table 5.
These results re-iterate the characteristics of these metrics to distinguish
batches of benign samples from adversarial samples.

Dataset | Density |
---|---|---
AE admixture | $0\%$ | $25\%$ | $50\%$ | $75\%$
MNIST | $1.007$ | $0.80$ | $0.59$ | $0.39$
PathMNIST | $0.83$ | $3.002$ | $4.92$ | $7.91$
OrganMNIST | $1.01$ | $1.41$ | $1.86$ | $2.29$
Table 4: _Density_ from batches of benign samples with proportionate admixtures of AEs for the boundary attack. Dataset | Coverage |
---|---|---
AE admixture | $0\%$ | $25\%$ | $50\%$ | $75\%$
MNIST | $0.14$ | $0.11$ | $0.07$ | $0.04$
PathMNIST | $0.49$ | $0.51$ | $0.56$ | $0.59$
OrganMNIST | $0.34$ | $0.30$ | $0.27$ | $0.25$
Table 5: _Coverage_ from batches of benign samples with proportionate
admixtures of AEs for the boundary attack.

##  Appendix D Runtime analysis

The analysis of runtime between prdc and our implementations is shown in
Figure 13. We compute the speed up, $\beta=\frac{T_{prdc}}{T_{n2gem}}$,
obtained in using our implementation in comparison to prdc. We have decreased
the computation time by nearly $100x$ when compared to prdc implementation.

Figure 13: The speed up in runtime between _prdc_ and our implementation.
Indicates the time for computing both _density_ and _coverage_ on a dataset
with specified sample size and $512$ dimensions.

## Review and Author response

In this section, we provide our response to the questions and concerns brought
forward by two reviewers during peer-review at the AdvML workshop with ICML
2022 where this preprint was submitted to. Let us first of all thank the
reviewers for investing their time in looking through our article in detail.

### Reviewer $1$

  1. 1.

”It seems that the FGSM attack and MNIST dataset adopted in this paper could
not reflect the effect of this method, maybe presenting the performance on
Cifar-10 under PGD attack will be more convincing.”
Our reply:

     * •

We have shown the results of the metrics _density_ and _coverage_ on the MNIST
dataset. To improve the complexity of the attack and the dataset used, we have
conducted experiments with $2$ additional and very different bio-medical
datasets, OrganMNIST and PathMNIST. To complement, we also used a black-box
attack and hoped that in light of the compute time expense that these two
attacks provide enough spectrum on the kind of attack.

     * •

The results indicated the capability of the metrics _density_ and _coverage_
to detect adversarial samples in all settings tested. The results in Section
5.2 discuss these observations.

     * •

In particular, Figure 3(a) addresses the concern by the reviewer. The text in
Section 5.2 discusses why we believe this to be evidence in favor of our
method.

  2. 2.

”In 3.2, the Density metric is unbounded, and the Coverage is bounded between
0 and 1. why are these two metrics different.”
Our reply:

     * •

As cited in Section 3.1, the density metric measures the number of benign-
sample manifolds containing an adversarial sample. Inherited in the
formulation of density, the metric’s value can surpass $1$, indicating that
the benign-sample neighborhoods and the adversarial samples are very closely
packed. Coverage, on the other hand, computes the fraction of benign-sample
hyperspheres containing at least one adversarial sample. Both metrics have
very different meanings, and in generative modeling, density demonstrates the
quality and coverage of the variability of the image for a given dataset.

  3. 3.

”Through the experiments, under FGSM, in Figure 3 (a) and Figure4, why does
PathMNIST express different trends following the changing of epsilon?”
Our reply:

     * •

The causes for the variation of _density_ and _coverage_ have been discussed
in Section 5.1. Furthermore, dimensionality reduction techniques also describe
the reason for such particular trends. The related analysis and results are
shown in the Appendix C.

  4. 4.

”Figure 3 should be modified to more clear, the legend, x/y-axis, and the line
in the figure are not clear.”
Our reply:

     * •

This is a valid statement. We could include an explanation of $\epsilon$ which
is given in the main text, but not in the caption.

     * •

We could add in the plot legends the word ”quantiles” (e.g.
$25\%-50\%\,\text{quantiles}$) to repeat the caption in the plot. We agree
that this may lead to a more quick visual digestion.

### Reviewer $2$

  1. 1.

”The chosen dataset is too simple and should be tested on at least the
CIFAR-10 dataset”
Our reply:

     * •

We focussed our introductory study on the MNIST dataset, which we acknowledge
as a valuable benchmark dataset to validate a few claims. However, we have
experimented with and validated our proposed metrics on $2$ very different
bio-medical datasets, which we believe are on par with the CIFAR-10 dataset.

     * •

In addition, we had hoped (but not documented in the text due to space
constraints) to explicitly not include too many established benchmark
datasets, as various authors Carter et al. (2021); Beyer et al. (2020) have
shown problems of overfitting on these in publications.

     * •

Last, the workshop website did not mention that using CIFAR-10 as a training
dataset is a requirement for a submission.

  2. 2.

”Why are medical datasets selected? In other words, I would like to know in
what scenarios medical images are attacked and need to be identified as
adversarial examples”
Our reply:

     * •

DNNs have been increasingly used in medical diagnostics and as a tool for
physicians in intricate fields of medicine such as radiology and
ophthalmology. Trillions of dollars worth of insurance claims are processed by
companies leveraging such automated systems. Not only focussing on financial
incentives but the vulnerability of algorithms should also be considered in
such a high stake field (Finlayson et al., 2019). We considered this situation
a ground model, concentrated on medical images, and diversified our validation
approach.

  3. 3.

”Limited novelty. For the problem this paper is trying to solve, the most
critical is metric, which has been proposed by Naeem et al.”
Our reply:

     * •

The AdvML2022 website clearly invites articles on ”Adversarial ML metrics and
their interconnections”

     * •

To our knowledge, _density_ and _coverage_ are metrics conceived for
characterizing the performance of GANs (Naeem et al., 2020).

     * •

As stated in Section 1, the novelty of our approach lies in the mapping of
metrics from the field of generative modeling to adversarial attacks.

     * •

Moreover, we have also worked to improve the runtime of the metrics in
comparison to the previous implementation, see Appendix D.

  4. 4.

”The validity of the proposed method needs to be verified with the latest
proposed attack”
Our reply:

     * •

In our study, we have experimented with both the extremes of adversarial
attacks, i.e., white- and black-box attacks. We believe that the proposed
metrics would scale well in detecting adversarial samples as our approach
remains model agnostic and only relies on the datasets in hand and the
generated adversarial samples.

     * •

Similar to our reply above, this appears a requirement to us which is not
stated on the AdvML2022 website.