# Adversarial Machine Learning in Wireless Communications using RF Data: A
Review ††thanks: This research work is supported by the U.S. Office of the
Under Secretary of Defense for Research and Engineering (OUSD(R&E)) under
agreement number FA8750-15-2-0119 and FA8750-15-2-0120.

Damilola Adesina†, Chung-Chu Hsieh‡, Yalin E. Sagduyu§, and Lijun Qian†
†Center of Excellence in Research and Education for Big Military Data
Intelligence (CREDIT Center)
Prairie View A&M University, Texas A&M University System, Prairie View, TX
77446, USA
‡ Center of Excellence in Cyber Security, Norfolk State University, Norfolk,
VA 23504, USA
§ Intelligent Automation, Inc., Rockville, MD 20855, USA
Email<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Machine learning (ML) provides effective means to learn from spectrum data and
solve complex tasks involved in wireless communications. Supported by recent
advances in computational resources and algorithmic designs, deep learning
(DL) has found success in performing various wireless communication tasks such
as signal recognition, spectrum sensing and waveform design. However, ML in
general and DL in particular have been found vulnerable to manipulations thus
giving rise to a field of study called adversarial machine learning (AML).
Although AML has been extensively studied in other data domains such as
computer vision and natural language processing, research for AML in the
wireless communications domain is still in its early stage. This paper
presents a comprehensive review of the latest research efforts focused on AML
in wireless communications while accounting for the unique characteristics of
wireless systems. First, the background of AML attacks on deep neural networks
is discussed and a taxonomy of AML attack types is provided. Various methods
of generating adversarial examples and attack mechanisms are also described.
In addition, an holistic survey of existing research on AML attacks for
various wireless communication problems as well as the corresponding defense
mechanisms in the wireless domain are presented. Finally, as new attacks and
defense techniques are developed, recent research trends and the overarching
future outlook for AML for next-generation wireless communications are
discussed.

###### Index Terms:

Adversarial machine learning, machine learning security, wireless security,
wireless attacks, defenses.

##  I Introduction

Machine learning (ML) in general and deep learning (DL) in particular has
found rich applications in various domains such as computer vision (CV) and
natural language processing (NLP). Motivated by the success in those domains,
there has been a growing recent interest in the development of artificial
intelligence driven solutions for wireless communications using radio
frequency (RF) data [1, 2, 3]. For example, convolutional neural network (CNN)
models have been used for modulation recognition [4] and channel decoding [5].
In addition to CNN, feed-forward neural network (FNN) and long short term
memory (LSTM) models have been used for device fingerprinting and
identification [6, 7, 8, 9]. On the other hand, various deep neural network
(DNN) models have been proposed for spectrum sensing and prediction [10],
wireless resource management [11], beam prediction for initial access [12] and
re-configurable intelligent surfaces [13]. Although the application of DL-
based techniques to wireless communications has shown promising results to
solve complex problems for which analytical solutions are either unavailable
or computationally infeasible, there are concerns in terms of reliability and
robustness of such data-driven methods when compared to traditional signal
processing methods in real-world deployments. One of such concerns is the
presence of adversaries that may target the training process and/or testing
(inference) process of DL.

With the increased adoption of DL, it is inevitable that adversaries will
explore launching new attacks particularly on DL models, thus leading to a new
paradigm called adversarial machine learning (AML). AML studies learning in
the presence of adversaries with the goal of understanding the impact of AML
attacks, defending against such attacks, and eventually developing secure DL
systems. One type of AML attack (namely adversarial attack or evasion attack)
occurs when an adversary crafts small perturbations to the original input of a
neural network or develop a means to manipulate the operation of the DL system
to cause an error in the inference process. These perturbations are not just
white noise samples but they are specifically crafted to produce a vector in
the input feature space that is capable of misleading the developed DL model.
Classical examples of such attacks include adding small incremental value in
the model’s gradient direction with respect to the inputs or by solving a
constrained optimization problem to produce a vector in the input feature
space that is capable of misleading the target DL model.

Figure 1: An example of adversarial attack in wireless communications. An
adversary adds perturbation to a test sample to create an adversarial example.
Then, the pre-trained ML model wrongly predicts the test sample.

AML attacks for CV and NLP have been extensively studied [14, 15, 16]. A
canonical example is the mis-classification of a panda image as gibbon because
of the addition of a specifically crafted perturbation to the panda image so
as to mislead the ML classifier [17]. Similarly, in the wireless domain,
adversarial attacks can lead to mis-classification of signals. As shown in
Figure 1, the pre-trained ML model is a classifier for automatic modulation
classification. An attacker launches an adversarial attack on the test data in
the inference phase by adding a small perturbation to the original QPSK test
data sample to form an adversarial example (AdEx). This AdEx causes the mis-
classification of a QPSK modulated signal as 16-QAM.

There have been multiple research efforts that aim to survey AML attacks as
well as recent advances to detect and mitigate them in specific areas where DL
has found applications such as CV and NLP [18, 19]. This review focuses on
approaches to generate, detect and mitigate AML attacks in the wireless domain
by accounting for the unique characteristics of wireless communications and
their effects on AML attacks.

###  I-A Unique Properties of Wireless Adversarial Attacks

Data-driven, intelligent solutions empowered by ML/DL are considered as the
key enablers in 5G and 6G wireless communications [20, 21, 22, 23]. DL has the
strong potential to assist network decisions in achieving optimal resource
management and future networks are envisioned to rely on DL-driven agents for
functions such as network management automation and optimization of radio
interfaces. The evaluation of the extent to which an adversary can affect such
a complex system becomes critical in designing future communication systems
with DL. As seen in [24, 25], the likelihood of jeopardizing ML/DL models
deployed to enhance systems is a challenge that can have dire consequences for
ML adoption. For instance, in cyberphysical systems such as autonomous
vehicles, adversarial attacks causing incorrect ML/DL decisions may harm lives
and properties [26, 27]. As such, to benefit from the gains of using ML/DL
systems, there is an urgent need to ensure security and robustness of ML-based
systems in the presence of adversarial attacks. An in-depth study of AML
specifically for wireless communication systems is ultimately needed given the
unique characteristics and emerging use cases of DL in such systems. Below, we
discuss these unique characteristics in the context of AML.

  * •

Effect of the communication channel: The wireless communication channel has a
major impact on adversarial attacks as they need to be launched over the air
to reach the target receivers that host the victim DL models. One aspect is
that the channel will impose path loss and phase on adversarial perturbations,
and could weaken and/or change the direction of adversarial signals as they
travel through wireless channels before reaching the target receivers. For an
adversarial attack to succeed over the air, the adversary needs to account for
the dynamic nature of channels when crafting the adversarial perturbations
[28, 29, 30, 31, 32]. In addition, the adversary may not have access to the DL
model at the target receiver and the process of data gathering by the
adversary for training a surrogate model is typically performed through a
channel, which implies that the training data used by the adversary is
imperfect by default and the effectiveness of the surrogate model trained by
the adversary strongly depends on channel effects [33]. Furthermore, it has
been shown in [34] that adversarial attacks crafted to fool DL models trained
on time-domain features do not necessarily transfer to DL models trained using
frequency-domain features. By leveraging differences in channels, it is also
viable to seek the objective of correct signal classification at one receiver
while fooling the signal classifier at another receiver [28, 35, 36, 31, 37].

  * •

Exploitation and mitigation of channel effects: The adversarial perturbation
may aim to fool a wireless signal classifier at one receiver while the
perturbed signal can be still decoded at the intended receiver with minimal
loss of reliability [36]. This paradigm can be used to hide wireless signals
(ranging from simple modulated signals to complex 5G signals) from
eavesdroppers [35]. While balancing the objectives of covertness and
communication performance, the adversary may aim to preserve not only the bit
error rate (BER) as in [31] but also the spectral shape of the perturbed
signal as in [38]. In addition, forward error correction (FEC) - a building
block in communication systems to detect and correct errors encountered due to
noise, channel effects, and hardware impairments - may reduce the effect of
adversarial attack while maintaining communication performance with the
intended receiver [39]. On the other hand, the adversary may use multiple
antennas to transmit perturbations to increase the effect of adversarial
attacks [30].

  * •

Indirect influence on training and test data: A wireless adversary cannot
directly manipulate the training or testing data input to a classifier. It
also cannot directly query a transmitter’s classifier and obtain its
classification results as opposed to the typical CV and NLP applications that
often rely on API queries. In an over-the-air attack, the wireless adversary
needs to monitor the actions in wireless communications and indirectly try to
manipulate/influence the outcome of the DL model or initiate actions such as
jamming the channel during sensing and data transmissions [40].

  * •

Heterogeneity in feature representation: The coexistence of various
communication systems introduces heterogeneity as a more diverse and complex
feature representation in radio data [41]. This significantly affects the
effectiveness of crafted perturbations. In this context, the adversary may
also aim to fool the wireless signal classifier at multiple receivers (each
possibly belonging to a different network) by transmitting a signal
perturbation and relying on the broadcast nature of omnidirectional
transmissions to reach and jointly affect multiple classifiers. Then, the
perturbation needs to be determined by accounting for multiple channels to
different receivers [28]. In addition to dynamic channels, it is also possible
that network traffic is of stochastic nature or traffic needs to be adapted to
dynamic channel effects. Then, the generation of perturbations needs to be
jointly considered with queue stability with the extended goal for the
adversary to reduce the stable throughput (the maximum achievable throughput
while keeping the packet queues stable [42]).

These unique properties necessitate the review of the current attacks and
mitigation methods of AML specifically for wireless communications as the
methods presented in other domains such as CV and NLP may not fit to the
characteristics of wireless medium. For instance, the adversary can directly
query an API for CV applications without the need to send perturbations over a
channel.

The design of DL algorithms for wireless communications must consider the new
security issues such as identification and mitigation of adversarial signals
[43]. Due to dynamic properties of RF data, it is a non-trivial task to
identify adversarial examples in RF data. DL methods rely on the premise that
the training data represents the distribution of the underlying data
generation process such that the trained model generalizes well on test data.
However, this premise does not necessarily apply in the adversarial scenarios
as these distributions change significantly due to adversarial effects that
are not straightforward to anticipate or predict [44, 45, 46, 47]. With such
adversarial attacks, wireless communications systems such as 5G and 6G with
design components based on DL will be susceptible to disruptions, performance
losses, and eventually failures. Therefore, it is critical to analyze the
emerging wireless attack surface due to AML, and reduce and ultimately
eliminate the impact of adversaries employing the AML attack techniques.

The rest of this paper is organized as follows. In Section II, previous works
on attacks on wireless communication systems as well as AML on other domains
are presented. Section III presents the background on AML detailing the
development of DNNs and adversarial perturbations. Section IV explores
different types of AML attacks and provides a taxonomy of AML attacks in the
wireless domain. Section V discusses the generation of AML attacks. The review
of the work on AML grouped under various wireless communication problem in
done in Section VI. In addition, AML attack detection and mitigation methods
are presented in Section VII while the future outlook of AML in wireless
communications is presented in Section VIII and Section IX the paper. The
symbols and notations as well as the definition of some important terms used
in this review are summarized in Section I-B and Section I-C respectively.

###  I-B Symbols and Notations

$X$
---
$x$
$x^{*}$
$Y$
$y$
$y^{\prime}$
$r_{x}$

total data points
---
a data point
adversarial example
class (label) for dataset
class for a data point
particular target label
adversarial perturbation

$w$
---
$J$
$\alpha$
$\epsilon$
$C$
$\bigtriangledown$
$f_{w}$

model parameters
---
cost function
step size
small positive number
number of classes
gradient operator
neural network classifier function

###  I-C Terms and Definitions

Terms |  Definitions
---|---
Adversary |  An agent that adds perturbation to clean data with the aim of fooling ML models.
Adversarial Perturbation |  The perturbation (noise or disturbance) signal generated by an adversary and added to clean data to alter it so as to fool an ML based system such as a classifier.
Adversarial Example |  A sample of the data that has been altered by adding a perturbation to fool an ML based system.
Adversarial Attack |  The process by which an adversary injects the adversarial example to undermine the ML-based system.
Adversarial Machine Learning |  Learning in the presence of adversaries with the goal of understanding the impact of AML attacks, defending against such attacks, and eventually developing secure DL systems.
Adversarial Training |  A defense mechanism where perturbed samples are added to the training data to boost model robustness against adversarial attacks.
Transferability |  The ability of an adversarial example to retain its potency when transferred between different models.
Robustness |  The ability of a model to remain reliable under changing conditions including adversarial attacks.

##  II Related work

DL has been extensively used in wireless communications as outlined in
numerous survey papers [1, 2, 3, 48, 49, 50, 51, 52, 53, 54, 55, 56]. These
papers highlight various models and applications of DL in wireless
communication systems. Specifically for AML in wireless communications, Table
I shows the types of DL models used in various related works.

TABLE I: DL models for Adversarial Machine Learning in Wireless Communications. DL Models |  Papers
---|---
Feedforward Neural Network |  [57, 29, 58, 59, 60, 61, 62]
Convolutional Neural Network |  [41, 63, 30, 38, 33, 39, 64, 60, 65, 66, 35, 31, 67, 68, 69, 70, 28, 32, 71, 72, 73, 40]
Recurrent Neural Network |  [74]
Reinforcement Learning |  [75, 76, 77, 78, 79]

There are various surveys in related areas, for instance, [80, 81, 82, 83, 84,
85, 86, 87, 88, 89] surveyed attacks on wireless communications systems, [57,
29, 58, 59, 60, 71, 72, 73, 38, 33, 39, 75, 76, 77, 78] used various ML models
for wireless communications while adversarial ML in other domains such as
computer vision and NLP was discussed in [90, 27, 91, 92, 25, 93, 18, 94, 95,
96, 97, 98, 99]. To the best of our knowledge, there is no thorough survey
available for AML in wireless communication systems.

Traditional attacks on communication systems such as primary user emulation,
jamming, eavesdropping attacks, and spectrum sensing data falsification have
been extensively studied in [100, 101, 102, 103, 104]. To further secure
communication systems, ML/DL techniques themselves can be used to detect
and/or mitigate such traditional attacks [105, 106, 107]. On the other hand,
AML has been reviewed and surveyed for various applications such as CV [18],
autonomous vehicles [27], and cyber security [108]. [109, 110, 95, 111, 112]
discussed the foundations of AML and provided detailed discussion on attack
and defense strategies as well as relevant perspectives for designing more
secure ML models. [113] focused on the taxonomy of attacks to identify areas
of action for researchers while [114] focused on adversarial example
generation and defense mechanisms. [99] provided a unified formulation for AML
on graph data and compared attacks methods and defense strategies.
Furthermore, [90, 96, 98, 97, 27, 115] provided various perspectives on AML in
the CV domain. For instance [98] focused on AML in object recognition, [97]
focused on AML in image classification and [27] focused on AML in autonomous
Vehicles. These works presented a taxonomy of attacks, defense strategies and
an outlook to the future. The scope of this review differs from previous works
as it explores AML attacks that directly target the growing ML/DL applications
in wireless communications. To the best of our knowledge, this is the first
review work that focuses on AML in wireless communications by discussing the
unique properties of wireless communications and its effects on AML. These AML
attacks are effective because they operate with low spectrum footprint, and
therefore they are stealthy (hard to detect) and energy-efficient  [26, 43,
77, 70, 72]. For instance, [58] showed that the DL-based exploratory attack is
more effective in reducing the system throughput and more energy efficient
than traditional random jamming attacks. The random jamming attack jams the
transmission channel in some randomly selected instances while the DL-based
exploratory attack jams in carefully selected time slots, thus using less
energy. Due to the unique properties of wireless communications, it is
essential to understand and quantify the impact of AML attacks on wireless
communications and develop defense mechanisms that are tailored for the
wireless domain.

##  III Background of Deep Adversarial Learning

###  III-A Deep Neural Network

Supervised learning using a DNN is expressed as a mapping $f$ that models the
mapping from the input data to a class (label). To learn this input-output
relationship, a weight matrix $w$ is estimated. A loss function $l(x,y,w)$ is
computed as a point-wise measure of error between the model prediction
$\hat{y}=f(x)$ and the observed ground truth $y$, where $x\in X$ is a sample
data point in data $X$ and $y\in\mathcal{C}$ is a class in $\mathcal{C}$. To
estimate $w$, a cost function $J(w)$, which is the average loss over all $(n)$
training data samples in $X$, is computed as [116]

${\color[rgb]{0,0,0}J(w)\equiv
J(X,y,w)=\frac{1}{n}\sum\limits_{(x_{i},y_{i})}l(x_{i},y_{i},w).}$ (1)

The model is derived by minimizing the cost function $J(w)$ as

$\operatorname*{argmin}_{w\epsilon\mathbb{R}^{n}}J(w).$ (2)

Then, the DNN classifier indexed by the DNN weights $w$ is determined as
$f_{w}:X\rightarrow\mathbb{R}^{C}$, where $C$ is the number of classes in
$\mathcal{C}$.

TABLE II: Summary of previous work on AML attacks in wireless communications. Categorization | Attack Type | Description | Paper
---|---|---|---
Attack Type | Exploratory | Fathom the inner workings of model | [58, 117, 57, 59, 61, 118, 119, 40, 75, 33, 77, 78]
Evasion | Fool ML models | [73, 65, 63, 71, 67, 41, 29, 59, 60, 57, 40, 30, 120, 35, 66, 28, 39, 38, 67, 74]
Causative | Manipulate the training process | [59, 40, 57]
Adversary’s Goal | Targeted | Cause specific errors | [73, 65, 29, 30, 35, 28, 33]
Non-Targeted | Cause any errors | [29, 32, 28, 70, 65, 74]
Amount of Knowledge | White-box | Full knowledge | [63, 29, 70, 60, 30, 35, 32, 28, 33, 74, 65]
Black-box | No knowledge | [41, 63, 29, 32, 60, 118, 73, 28, 76, 79, 61, 62]
Gray-box | Limited knowledge | [64, 76]

###  III-B Adversarial Attack on Deep Neural Network

As an example of AML attacks, we describe next how the evasion or the so-
called adversarial attack works. We will discuss other types of AML attacks in
Section IV. For every $x\in X$, $f_{w}(x)$ assigns a label of the form
$\hat{y}=\operatorname*{argmax}_{y\in\mathcal{C}}f_{w}(x,y)$, where
$f_{w}(x,y)$ corresponds to the output of $f_{w}$ for class $y$ in
$\mathcal{C}$. An adversarial attack on the classifier $f_{w}$ aims to modify
the classified input $x$ with perturbation $r_{x}$ such that
$f_{w}(x+r_{x})\neq f_{w}(x)$. For the stealth nature of this attack, it is
required that the perturbation $r_{x}$ should be small such that the
difference of $x+r_{x}$ from the unperturbed input data $x$ remains small.
Typically, this requirement is relaxed to constraint
$||r_{x}||_{p}\leq\epsilon$, where $\epsilon$ is a small positive number and
$||\cdot||_{p}$ represents the $l_{p}$ norm with $p\in\\{1,2,\infty\\}$
denoting the type of norm. The adversarial perturbation $r_{x}$ for $x$ and
$f_{w}$ is determined by solving the following optimization problem:

$\displaystyle\min\limits_{r_{x}}$ $\displaystyle||r_{x}||_{p}$ $\displaystyle
s.t.$ $\displaystyle f_{w}(x)\neq f_{w}(x+r_{x}).$

In the context of wireless communications, the $l_{2}$-norm is a natural
choice for $r_{x}$ as it accounts for the perturbation (signal) power [63]
[70] compared to other norms such as the $l_{1}$-norm that has been used in
the CV domain to deter the variation from human perception [70].

##  IV Categorization of Adversarial Machine Learning Attacks on Wireless
Communications

We present a categorization of AML attacks to aid the understanding of key
concepts of AML and provide an overview of AML attacks from various
perspectives borrowed from [57, 59, 29, 71, 121, 58, 122, 41] for wireless
communications and [91, 123, 93, 95, 124, 94, 25, 112] for other domains
especially CV. In particular, we discuss adversarial attacks in wireless
communications under three main categories (see Fig. 2) that are defined
according to the influence - the type of attack, the attack phase, and the
amount of knowledge the adversary has about the victim model. Table II lists
previous works of AML attacks on wireless communications under these
categories.

Figure 2: Categorization of adversarial machine learning.

###  IV-A Categorization Based on Influence - Types of Attack

  * •

Exploratory attacks: Also called inference attacks, exploratory attacks seek
to fathom the inner workings of ML algorithms/models by collecting training
data and imitating the ML model functionally with similar types of inputs and
outputs, namely building a surrogate (shadow) model [125, 126]. The
exploratory attack is typically the leading step prior to subsequent attacks
as it aims to “explore” the victim model using techniques such as active
learning [127] or augment the limited information with generative adversarial
networks (GANs) [128]. Adversarial attacks designed with the surrogate model
are known to transfer to the target model (known as the transferability
property [129]). In a wireless communications scenario, the adversary can
learn the transmission patterns of the victim communication system by
observing the spectrum over the air. The effect of wireless channels on
surrogate models was studied in [33].

  * •

Evasion attack: Also called adversarial attacks, evasion attacks aim to fool
ML models into making wrong decisions by manipulating the input test data [14,
17]. Evasion attacks have been applied to wireless communications in terms of
fooling classifiers used for spectrum sensing [40], modulation recognition
[63], autoencoder-based end-to-end communication systems [60], and channel
state information (CSI) feedback for massive MIMO [70, 130] and initial access
in directional communications [131]. One challenge in the wireless domain is
that the adversarial perturbations observed by the target classifier are
received subject to channel effects such that these perturbations need to be
crafted by accounting for channel effects [28, 31, 32, 72]. These attacks are
typically stealthier and more energy-efficient than traditional jamming
attacks (such as the one that solely aim to cause interference to data
transmission, e.g., [132, 103, 104]) as they only need to transmit low-power
signals over a short period of time to confuse the ML algorithms in their
decision making.

  * •

Causative attack: Also called poisoning attacks, causative attacks aim to
manipulate the training process of ML models by injecting vulnerabilities such
as false training data to the ML models [133, 134]. Causative attacks are
effective against DNNs as they are highly susceptible to inaccuracies in
training data. An example of a causative attack in wireless communications is
spectrum poisoning where the adversary exploits the (re)training process of
the ML classifier so that the ML model is poorly (re)trained. Examples of
causative attacks in wireless communications include attacks on spectrum
sensing [40], cooperative spectrum sensing [118], and IoT systems [119, 135].

Figure 3: Attacks in adversarial machine learning [136].

  * •

Trojan attacks: Also called backdoor attacks are combinations of evasion and
causative attacks, where the adversary injects triggers (backdoors) to
training data and then activates them for some input samples in test time
[137, 138]. Trojan attack has been considered against wireless signal
classifiers in [69] by formulating controlled phase shifts as backdoors and
adding them into the transmitted and received RF data samples.

Figure 3 gives a pictorial representation of AML attack models and shows how
the different attacks work, as well as the general procedure for attack
formation.

###  IV-B Categorization Based on Attack Phase.

  * •

Training phase: The causative and trojan attacks occur in the training phase
of the model development. See Section IV-A for discussion on these attacks.

  * •

Testing phase: The exploratory attack, evasion attack and trojan attack occur
in the test time. See Section IV-A for discussion on these attacks.

###  IV-C Categorization Based on The Amount of Knowledge The Adversary Has
About The Victim Model

  * •

White-box attack: The adversary knows the training data, architecture,
algorithm and/or optimization techniques such that it has full access to the
trained model $f$ and knows the input at the classifier. For an over-the-air
white-box attack in wireless communications, the adversary also needs to know
the channel between the adversary and the receiver, since the channel affects
the perturbation perceived at the receiver [28, 31, 32, 33].

  * •

Black-box attack: The black-box attack is a more realistic and more relaxed
model for many security threats [139] as the adversary has neither knowledge
about nor access to the training data and/or the trained model $f$. In that
case, the adversary tries to deduce information from the returned results of
the model. Black-box attacks typically use a surrogate (shadow) model, which
is trained to perform an identical task as the target network during the
inference attack. In wireless communications, the surrogate model strongly
depends on the channel characteristics with respect to the adversary and the
victim system [33].

  * •

Gray-box attack: In a gray-box attack, the adversary has some knowledge of the
data/algorithm as well as limited access to the model $f$. One example from
the wireless domain is not knowing the exact channel but knowing an estimate,
the distribution or some statistics of the channel, e.g., Rayleigh vs. Rician
fading, the value of pathloss exponent or the value of signal-to-noise-ratio
(SNR), etc. [28].

###  IV-D Categorization Based on Type of Adversary’s Goals

  * •

Targeted attack: The adversary tries to produce the perturbation $r_{x}$ that
will cause specific errors in the output of a receiver by targeting a signal
$x$ to generate errors such that $f_{w}(x+r_{x})=y^{\prime}$ for a particular
target class $y^{\prime}$. For example, consider a 4-class modulation
classification problem with BPSK, QPSK, QAM 16 and QAM 64 as classes and
consider the adversary that seeks to fool the modulation classifier into
classifying a signal modulated with QAM 16 as a QAM 64.

  * •

Non-targeted attack: The adversary tries to produce the perturbation $r_{x}$
that will cause errors in the output of the algorithm independent of the
classes thereby reducing the confidence in the algorithm due to decreased
accuracy. In our previous example of a 4-class modulation classification
problem with BPSK, QPSK, QAM 16 and QAM 64 as classes, the adversary seeks to
fool the modulation classifier into classifying a signal modulated with QAM 16
as any other available class such as BPSK, QPSK, or QAM 64, i.e.,
$f_{w}(x+r_{x})\neq y$ for any $y=f(x)$.

###  IV-E Other Types of Categorization

Some other types of categorization for AML attacks include categorization
based on the frequency of attack and categorization based on the degree of
freedom the adversary has in accessing the system’s input.

  * •

In the attack frequency category [41], there is a one-step attack that
computes the gradient of the loss function at a time to create perturbations.
There is also an iterative attack that performs the same computation as a one-
step attack multiple times to create the perturbations. This is enabled by
perpetual access to the model and can be computationally intensive and costly.

  * •

Based on the adversary’s degree of freedom to the input of the system, there
may be physical and digital attacks. In a physical attack, there is an
indirect application of the input to the model which is different from the
digital attack where the adversary can explicitly design the input of the
model [60] [139]. We can also remove the requirement for the adversary to know
the input test samples and craft input-agnostic adversarial attacks, also
called Universal Adversarial Perturbation (UAP) [140]. UAP attacks can be
built against wireless signal classifiers, where the adversary does not need
to know about the transmitted signal [63, 28]. It is also possible to limit
attacks to certain users, e.g., evasion attacks to fool classifiers only at a
subset of receivers [28] and causative attacks launched from a selected subset
of IoT devices to manipulate the training process at a fusion center [119].

These attack types may overlap in the description of an attack. For example,
the authors in [29] studied both targeted, white-box evasion attack and non-
targeted white-box evasion attacks for DL-based modulation classification.

##  V Adversarial Machine Learning Attacks in Wireless Communications

In this section, we describe the approaches to generate attacks in AML,
including evasion attacks, exploratory attacks and causative attacks.

###  V-A Generation of Evasion Attacks

Evasion attacks to generate adversarial examples are characterized as subtly
crafted imperceptible perturbations $r_{x}$ added to the original inputs $x$
of the ML model such that the input to the classifier is $x^{\prime}=x+r_{x}$
by solving the optimization problem (III-B) for $r_{x}$. In the wireless
domain, the problem is changed by setting with
$x^{\prime}=h_{t,r}x+h_{t,a}r_{x}+n$, where $x^{\prime}$ is the received
signal, $x$ is the transmitted signal, $r_{x}$ is the transmitted
perturbation, $h_{t,r}$ is the channel gain from the transmitter to the
receiver, $h_{t,a}$ is the channel gain from the transmitter to the adversary,
and $n$ is the receiver noise.

It is difficult to solve (III-B) due to the non-linearity of the DNN mapping.
Instead, we can approximate the solution. This is achieved by maximizing the
loss function $L(w,x,y)$ used in the training process of the classifier
$f_{w}$ (e.g., the cross-entropy loss function is
$L(w,x,y)=-\log(1+\exp({-f_{w}(x,y)}))$ subject to the condition that the
perturbation is upper bounded. Thus, the adversarial perturbation $r_{x}$ can
be found by solving

$\displaystyle\max$ $\displaystyle L(w,x+r_{x},y)$ $\displaystyle s.t.$
$\displaystyle\min||r_{x}||_{p}\leq\epsilon,$

where $\epsilon$ is the upper bound on the adversarial perturbation. Various
methods have been proposed to approximately solve (V-A), thus generating
adversarial examples. These methods include but are not limited to one-step
gradient-based method such as Fast Gradient Sign Method (FGSM) [17], and its
iterative variants such as Basic Iterative Method (BIM) [139], Projected
Gradient Descent (PGD) [141], Momentum Iterative Method [142], Box Constrained
Limited-Memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) [14], and Carlini &
Wagner (C&W). We describe some of these methods that have been used to
generate adversarial attacks on wireless communications.

####  V-A1 Fast Gradient Sign Method (FGSM) [17]

One popular method that has been used to generate adversarial examples in the
wireless domain is the FGSM. The FGSM is a one-step gradient-based method
developed on finding the scaled sign of the gradient of the cost function and
aims at minimizing the strength of the perturbation. It is a computationally
effective method for adversarial attack generation based on the postulation
that the DNNs are prone to linear-type adversarial attacks because of their
use of linear techniques for easier optimization. The FGSM-generated
perturbation is achieved by linearizing the model’s cost function $J(w,x,y)$
around the current value of $x$. FGSM generates an adversarial example that
significantly resembles the original input $x$ by starting with an original
input to the ML model and adjusts it in the direction of the gradient of the
loss function with respect to the input by an amount limited by a parameter
$\epsilon$ stated as:

$x^{*}=x+\epsilon\cdot\text{sign}{\color[rgb]{0,0,0}(}\nabla_{x}J(x,y,w)),$
(5)

where $x^{*}$ is the crafted adversarial example, $x$ is the original input,
$J(x,y,w)$ is the cost function of the DNN with parameters $w$, $y$ is the
class associated with $x$, and $\epsilon$ is a small number to limit the
perturbation [120]. Note that the Fast Gradient Method (FGM) is a
generalization of FGSM to meet the $L_{2}~{}$norm bound
${||x^{*}-x||}_{2}<\epsilon$ as

$x^{*}=x+\epsilon\cdot\frac{\nabla_{x}J(x,y,w)}{{{||\nabla}_{x}J(x,y,w)||}_{2}}.$
(6)

FGSM is also used in iterative methods such as the following.

  * •

Basic iterative method (BIM) [139]: The BIM can be viewed as multiple steps of
FGSM with a small step size as well as a clipping of results after each
iteration to establish that the results are in an $\epsilon$-neighborhood of
the original input data. The iteration for the BIM is given by

$x_{i}^{*}=\text{clip}_{x,\epsilon}\left(x_{i-1}^{*}+\epsilon\cdot\text{sign}(\nabla_{x}J({x^{*}_{i-1}},y,w))\right),$
(7)

where $\text{clip}_{x,\epsilon}$ clips the values of the adversarial sample to
an $\epsilon$-neighborhood of the original sample $x$.

  * •

Projected Gradient Descent (PGD) [141]: PGD is a multi-step variant of FGSM on
the negative loss function that helps initiate adversarial attacks with the
aim of examining or analyzing the performance of a neural network from an
optimization point of view. It puts a limit on the total perturbation in
$L_{\infty}$-norm from $x^{*}_{0}=x+U_{n}(-\epsilon,\epsilon)$, where
$U_{n}(-\epsilon,\epsilon)$ is the uniform random perturbation. In $t$
iterations, $x$ is determined as

$x^{*}_{t}={\Pi}_{B_{\epsilon}(x)}\left(x^{*}_{t-1}+\epsilon\cdot\text{sign}\left(\nabla_{x}J(x^{*}_{t-1},y,w)\right)\right),$
(8)

where ${\Pi}_{B_{\epsilon}(x)}$ is the projection operator. The procedure of
adversarial example generation includes adding noise, computing gradient,
stepping, and projecting back.

  * •

Momentum Iterative Method (MIM) [142]: The MIM was developed to solve the
problems related to over-fitting and local minima. It helps speed up the
gradient descent iterations. In scenarios where the performance of FGSM is
inhibited due to the presence of strong interference and distortion, the MIM
applies momentum to the iterative attacks so as to stabilize and maintain the
correct direction of the update with generalization of adversarial examples.

####  V-A2 L-BFGS Method [14]

Box-constrained L-BFGS is an example of optimization-based method for
generating adversarial examples. For an instance of $x$, L-BFGS finds a
different $x^{*}$ that is very much alike to $x$ based on the $L_{2}$ distance
but is labeled differently by the classifier. This problem is laborious and is
modeled as the following constrained minimization:

$\displaystyle\min$ $\displaystyle{||x-x^{*}||}_{2}^{2},$ $\displaystyle s.t.$
$\displaystyle f_{w}(x+r_{x})=y^{\prime},$
$\displaystyle(x+r_{x})\in[0,1]^{n},$

where $y^{\prime}$ is the target class. To solve (V-A2), it is transformed to

$\displaystyle\min$
$\displaystyle\lambda\cdot{||x-x^{*}||}_{2}^{2}+J(x,y^{\prime},w)$
$\displaystyle s.t.$ $\displaystyle x^{*}\in[0,1]^{n},$

where $J(x,y^{\prime},w)$ is a function mapping an instance of $x$ to a
positive real number using the loss function. In (V-A2), the cross-entropy
loss function is typically used as the loss function $J$ and line search is
used to find the positive constant $\lambda$ that generates an adversarial
example of minimum distance.

###  V-B Generation of Exploratory Attack

In an exploratory attack, an adversary seeks to understand the internal
operations on an ML model and leverage on such knowledge to attack the ML
model and ultimately undermine the effectiveness of the ML-based system. This
is achieved by developing a surrogate (or shadow) model that can mimic the
operation of the original ML model. The approach is to steal training data and
build models that have the same (or similar) output as the original model.
With the information on how the original model functions, the attacker can
make informed decisions as to how to attack and alter the overall performance
of the ML-based system. For example, in [117], an exploratory attack is
launched on the cognitive radio as a preliminary step to learn when and how to
jam the cognitive radio transmissions. The adversary senses a channel,
captures the transmitter’s decisions and learns the pattern of successful
transmissions by tracking acknowledgments from the receiver, and launches an
exploratory attack to build a DL classifier (surrogate model) that is
functionally equivalent to the one at that transmitter. In another example
[57], an adversary observes the spectrum and develops a DNN to infer the
channel access algorithm used by an IoT transmitter and predict the outcome of
the transmissions before jamming it. Exploratory attacks are also launched to
infer reinforcement learning mechanisms used in wireless channel access and
jam the underlying communications.  [75, 76] designed a DRL-based jamming
attacker that employs a dynamic policy and aims to minimize the channel access
accuracy of a DRL-based dynamic channel access user. The DRL attacker is able
to observe the victim’s interaction with its environment for a period of time
and learns the activity pattern. The attacker has no prior information about
the channel switching pattern on a victim’s action policy and both DRL-based
systems can interact with each other, retrain their models and adapt to the
opponent’s policy. Also, [77, 78] introduced a reinforcement learning (RL)
algorithm to disrupt 5G network slicing by observing the spectrum and
developing an RL-based surrogate model. This model decides which resource
blocks to jam in order to attain a high number of failed network slicing
requests. Jamming a resource block causes a reduction in the RL algorithm’s
reward thus, affecting the model’s performance as the reward is used to update
the RL algorithm.

Although the exploratory attacks described above follow the same paradigm,
there are no standard attack generation methods as we have in evasion attacks.
All the methods use a common approach based on learning the internal workings
of the original model, building a surrogate model and attacking the ML-based
model using the knowledge from the surrogate model. Table II shows examples of
previous works on exploratory attacks.

###  V-C Generation of Causative Attack

The causative attack seeks to downgrade the ability of a ML system to perform
optimally by injecting vulnerabilities into the training process. These
injected vulnerabilities can be as a result of the adversary by intentionally
meddling with the label of the data or by crafting and embedding unique
learnable features. The model learns these unique features which causes it to
deviate from its original objective, thus, causing considerable degradation in
the ability of the model to make good predictions. The injection of
vulnerabilities might be during the initial model development or the model re-
training process. Since the attack occurs before model training, the
contamination affects any type of ML model and efforts on parameter tuning
yields little or no improvement on the model’s ability to make good
predictions [136]. As we have observed in the exploratory attack, there are no
standard attack generation methods but the paradigm of poisoning the training
data is consistent in all forms of causative attacks. Table II shows some
previous work on causative attack.

##  VI Review of Adversarial Machine Learning in Wireless Communications

Grouped according to the type of DL problem, we discuss and review various
studies on AML in wireless communication and present them in Tables III (for
modulation classification), Table IV (for spectrum sensing), Table V (for
signal classification), and Table VI (for other areas in the wireless
communication domain).

###  VI-A Modulation Classification and Signal Classification:

Modulation classification – the process between the detection and demodulation
of a signal – is an important step towards developing an intelligent radio
receiver and has been extensively studied [143]. Recently, the research in
this area has gained new attention with the use of DL. [144] showed that DL
algorithms for modulation classification using RF data outperform the use of
expert features and higher order moments. AML for modulation recognition has
been extensively using evasion and/or trojan attacks. [63, 41, 66] showed the
susceptibility of DL models to adversarial attacks on modulation
classification under different scenarios such as data with noise and variation
in SNR level. [63, 71] pointed out that AdExs for modulation classification
are more effective than perturbations crafted by adding Gaussian noise. Table
V shows AML studies focused on modulation classification. Similar to the
modulation classification problem, AML has been also applied to wireless
signal classification. Signal classification aims to identify the transmitting
signal in a spectrum. Table III highlights relevant work on AML for signal
classification.

TABLE III: Review of previous works on AML attacks in modulation classification. Paper |  Attack Type |  Objective |  Attack Method |  DL Problem |  Data used
---|---|---|---|---|---
[29] |  White-box, Black-box, Targeted, Untargeted, Evasion |  Explore how to launch a realistic evasion attack by taking into consideration the channel effect as well as the power constraint at the adversary. |  FGM, UAP |  Modulation classification |  RML2016.10A [145]
[66] |  Evasion |  Test the attack efficiency of two types of adversarial attacks. |  FGSM, Box-constrained L-BFGS |  Modulation classification |  RML2016.04C [4]
[41] |  Evasion, Black-box, White-box |  Explore the feasibility and effectiveness of adversarial attacks and determine the optimal perturbation level for attack invisibility and success. |  FGSM, PGD, BIM, MIM |  Modulation classification |  RML2016.10A [145]
[69] |  Trojan attack |  Introduce a new attack that slightly manipulates training data by inserting Trojans (i.e., triggers) to a few training data samples and then activate them later in test time. |  Embed Trojans in training data and trigger it in test time |  Modulation classification |  RML2016.10A [145]
[71] |  Evasion |  Evaluate the vulnerabilities of raw I/Q based modulation classification. |  FGSM |  Modulation classification |  RML2016.10A [145], three synthetic datasets using GNU radio
[36, 31, 146] |  Evasion |  Minimize the accuracy of the intruder in determining the modulation scheme used by a transmitter by perturbing the channel input symbols. |  PGD |  Modulation classification |  Locally generated data
[64] |  Gray-box |  Mitigate adversarial examples in RF classifiers using autoencoder preprocessing. |  FGSM |  Modulation classification |  RML2018A [147]
[30] |  Evasion White-box Targeted |  Investigate the use of multiple antennas to generate multiple concurrent perturbations over different channel effects (subject to total power budget) to the input of DNN-based modulation classifier at a wireless receiver. |  FGM |  Modulation classification |  RML2016.10A [145]
[63] |  Evasion, White-box, Black-box |  Show the susceptibility of DL models to adversarial attacks and present practical methods for crafting adversarial examples in modulation classification. |  PCA-based UAP, FGM-modified bisection search |  Modulation classification |  RML2016.10A [145]
[35] |  Evasion, Targeted, White-box |  Fool a DL-based eavesdropper by a cooperative jammer transmitting adversarial perturbation so as to hide 5G communications from the eavesdropper. |  FGM |  Modulation classification and 5G system |  Locally generated data using MATLAB 5G toolbox
[39] |  Evasion |  Extend the use of communication-aware evasion attack through the use of forward error correction (FEC). |  Perturbation created by Adversarial Mutation Network |  Modulation classification |  Synthetic data generated using Liquid DSP
[65] |  Evasion, Untargeted, White-box, Targeted |  Examine how the DL-based automatic modulation classifier breaks down in the presence of an adversary with direct access to its inputs. |  FGSM, MI-FGSM |  Modulation classification |  RML2016.10A [145].
[28] |  Evasion, Black-box, Untargeted, White-box, Targeted |  Explore how to design realistic adversarial attacks in the presence of realistic channel effects and multiple classifiers at different receivers. |  FGM, UAP |  Modulation classification |  RML2016.10A [145]
[32] |  White-box, Untargeted Targeted |  Provide a generalized wireless AML problem (GWAP) and experimental evaluation of AML attacks launched against wireless DL systems |  AML waveform jamming, AML waveform synthesis |  Modulation classification, Radio Fingerprinting |  RML2018.01A [147], 1000-device dataset of WiFi and ADS-B transmissions
[38] |  Evasion |  Introduce spectral deception loss metric implemented during training to make the spectral shape more in-line with the original signal. |  Adversarial Mutation Network (AMN) |  Modulation classification |  Locally generated data
[67] |  Evasion |  Minimize the accuracy of an intruder while still ensuring the successful decoding of a signal by the intended receiver. |  PGD, uniform random noise of $L_{2}$-norm to a block of $128$ I/Q symbols |  Defense against modulation detection |  Locally generated data
[148] |  Evasion |  Study the effect of adversarial attacks on ML-based AMC model using different data-driven subsampling strategies |  Carlini-Wagner |  Modulation classification |  RML2016.10B [149]
[150] |  white-box, black-box |  Present an input-agnostic adversarial attack that is undetectable and robust to removal |  UAP |  Modulation classification |  RML2016.10A [145]

###  VI-B Spectrum Sensing:

Most of the AML studies focused on spectrum sensing and dynamic spectrum
access (DSA) use exploratory attacks in which the attacker learns the
operational pattern of the DNN model and builds a surrogate model which gives
the same or similar output as the original classifier when given the same
input. Spectrum sensing and DSA are key components of cognitive radio systems
for efficient discovery and use of spectrum to achieve situational awareness
[59]. Given the broadcast and over-the-air nature of wireless communications,
there are various attacks that attempt to undermine the spectrum sensing
results. This can include causative attack to change the sensed features as
well as priority violation attacks to violate priorities in channel access by
pretending to have higher priority and transmitting during the sensing phase
[57]. AML attacks on spectrum sensing typically involve an adversary that
develops a surrogate model and can correctly predict the outcome of the
original DL model. Such an adversary jams the successful transmissions, thus
reducing the success rate of the original model [58, 117, 75]. As shown in
[58, 117, 40, 118], these attacks are more energy-efficient and more effective
than random jamming. Table IV highlight the studies on AML for spectrum
sensing and DSA applications.

TABLE IV: Review of previous works on AML attacks in spectrum sensing. Paper |  Attack Type |  Objective |  Attack Method |  DL Problem |  Data used
---|---|---|---|---|---
[58, 117] |  Exploratory |  Launch an exploratory attack on cognitive radio as a preliminary step before jamming. |  Jam legitimate user transmissions |  Cognitive radio channel access |  Locally generated data
[59] |  Spectrum poisoning, Exploratory |  Launch an over-the-air attack to infer the transmitter’s behavior and falsify the spectrum sensing data by transmitting perturbations over the air. |  Inject perturbation to the channel |  Spectrum sensing |  Locally generated data
[57] |  Exploratory, Evasion, Causative, Black box |  Build a DNN classifier to infer the channel access algorithm of a transmitter and predict the transmission result, and develop a defense mechanism to confuse the adversary in its attack strategy. |  Jamming, Spectrum poisoning, Priority violation |  Spectrum sensing |  Locally generated data
[118] |  Black-box, Exploratory |  Propose Learning-Evaluation-Beating(LEB) attack against cooperative spectrum sensing (extension of spectrum sensing data falsification attack on cognitive radio). |  LEB attack |  Cooperative spectrum sensing |  Dataset collected from 5282 locations [151], Data from 5 locations using USRP N210
[76] |  Black box Gray box |  Develop deep reinforcement learning (DRL)-based dynamic channel access system and DRL-based jamming attack to study the sensitivity of DRL-based wireless communications to adversarial attacks. |  DRL-based attacker use dynamic policy to minimize the victim’s channel access |  Dynamic spectrum access (DSA) |  Locally generated data.
[40] |  Exploratory, Evasion, Causative |  Launch over-the-air spectrum poisoning attacks by learning first the transmitter’s behavior. |  Attacker jams the transmitter’s sensing period |  Spectrum sensing, DSA |  Locally generated data
[75] |  Exploratory |  Compare the performances of two adversarial policies: feed-forward neural network (FNN) and DRL policies on the performance of a DRL-based dynamic channel access agent. |  Jamming of a victim model |  Dynamic spectrum access |  Locally generated data
[152] |  Exploratory, Evasion |  Present AML attack on spectrum sharing of 5G communications with incumbent users. |  Manipulate inputs to the classifier by transmitting during spectrum sensing |  Spectrum sharing |  Locally generated data

###  VI-C Other Areas in Wireless Communication:

AML attacks have been also applied to other areas in wireless communications.
This includes resource allocation such as network slicing [77, 78], IoT data
fusion process [119], waveform/communications as in power control [130], beam
alignment and initial access [131]. A review of AML studies on other areas in
wireless communication is presented in Table VI.

TABLE V: Review of previous works on AML attacks on signal classification. Paper |  Attack Type |  Objective |  Attack Method |  DL Problem |  Data used
---|---|---|---|---|---
[61] |  Black-box, Exploratory |  Show that DL-based signal classifiers are vulnerable to privacy threats due to over-the-air information leakage of their model. |  Membership inference attack (MIA) |  Wireless signal classification |  Locally generated data
[33] |  Exploratory, White-box, Targeted |  Investigate the channel effects on surrogate model built by an adversary that uses the over-the-air observed spectrum data. |  Maximum Received Perturbation Power (MRPP) attack [29] |  Wireless signal classification |  RML2016.10A [145] & locally generated data
[120] |  Evasion |  Develop methods for detection of adversarial perturbations used against wireless signal classifiers. |  FGSM |  Survey application & spectrum sensing |  RML2018A [147], over-the-air Bluetooth, WiFi, ZigBee datasets
[74] |  Evasion, Untargeted, White-box |  Fool a DL-based signal classifier (authenticator). |  Reinforcement learning-based attack |  Signal authentication |  Simulated data and testbed data from Pluto SDRs
[73] |  Targeted, Black box |  Analyze the effect of adversarial examples against Deep RF classifiers and also develop several defense mechanisms. |  Carlini-Wagner attack |  Signal & , protocol classification |  Locally generated WiFi, Bluetooth & ZigBee data. RML2018A [147]
[150] |  white-box, black-box |  Present an input-agnostic adversarial attack that is undetectable and robust to removal. |  UAP |  Signal Detection in OFDM Systems |  RML2016.10A [145]
TABLE VI: Review of other previous works on AML attacks in wireless communications Paper |  Attack Type |  Objective |  Attack Method |  DL Problem |  Data used
---|---|---|---|---|---
[60] |  Black-box, Exploratory, Evasion |  Show that end-to-end learning of communication systems through autoencoder can be extremely vulnerable to physical adversarial attacks. |  Iterative method based on UAP |  End-to-end autoencoder communication systems |  Locally generated data
[119] |  Exploratory |  Employ AML techniques to launch attacks against the IoT data fusion process. |  Partial-model attack |  IoT data fusion/aggregation process |  Synthetic data from Gaussian distributions
[70] |  White-box, Untargeted |  Present a method to craft adversarial attack and show its effect on DL-based channel state information (CSI) feedback process. |  Attacker modeled by a bias layer between the encoder and decoder of CsiNet |  Massive MIMO CSI feedback process |  Locally generated data generated using COST 2100 channel model [153]
[79] |  Black-box |  Develop defense strategies against DRL-based jamming attacks on a DRL-based dynamic multichannel access agent. |  DRL-based jamming attacker |  Multichannel access |  Locally generated data
[77] |  Exploratory |  Present an over-the-air attack to manipulate the RL algorithm and disrupt 5G network slicing. |  Adversary builds an RL-based model and jams selected resource blocks |  Resource allocation for 5G network slicing |  Locally generated data
[78] |  Exploratory |  Present flooding attack on 5G network slicing. |  Adversary crafts fake network slicing requests to consume the 5G radio access network resources |  Resource allocation for 5G network slicing |  Locally generated data
[152] |  Exploratory, Spoofing |  Present adversarial attack on signal authentication in network slicing. |  Spoofing attack on the network slicing application |  Physical layer authentication of 5G device |  Locally generated data
[131] |  non-targeted |  Investigate the vulnerability of a DNN used for mmWave beam prediction as part of the initial access process in 5G and beyond communications. |  FGM, adversary tries to change the beam to one of the worst beams |  mmWave beam prediction in 5G |  Locally generated data
[130] |  white-box, black-box |  Show that adversarial attacks can break DL-based power allocation in the downlink of a massive multiple-input-multiple-output (maMIMO) network. |  (FGSM), momentum iterative FGSM, and PGD |  Power allocation in a maMIMO |  DL power allocation in massive MIMO dataset [154]

##  VII Detection and Mitigation of AML Attacks in Wireless Communications

Next, we present various methods that have been used to defend against AML
attacks on wireless communication systems.

###  VII-A Train with Adversarial Perturbations

Training the DNNs with adversarial examples is a popular method of mitigating
the effect of adversarial attacks and ensuring robustness [141]. Adversarial
training consists of generating adversarial examples according to one or more
attack methods and retraining the DNNs with labeled adversarial examples. The
goal is to prevent an adversary from identifying the structure of the signals
transmitted so as to prevent it from disrupting the communication system. This
defense mechanism has been applied to protect wireless signal classifiers
against evasion attacks [28], where randomized smoothing [155] used in
training time increases the robustness of classifiers against adversarial
attacks later in test time.

There are some challenges that pertain to adversarial training; some of these
include a scenario where the adversary uses a different attack from the one
used in training. In addition to that, an adversary can create new
perturbations using a model trained with adversarial examples. However,
training with adversarial examples typically reduces the performance of DL
models on unperturbed signals. In trying to overcome these challenges,
training can be performed with multiple adversarial examples [141]. In
addition, certified defense [156] can be built in train and test times to
ensure statistical significance of classification results in the presence of
adversarial attacks in test time. Certified defense has been used in [28] to
provide performance guarantees for wireless signal classifiers in the presence
of adversarial examples. Another way to mitigate adversarial attacks during
training is pre-training of the DL based wireless signal classifiers by an
autoencoder such that the trained model becomes robust to the deceiving effect
of adversarial examples [64].

###  VII-B Statistical Approaches

  * •

Peak-to-average power ratio (PAPR) of RF samples: One way of detecting
adversarial examples it to leverage the properties of radio frequency (RF)
data and apply a statistical test by taking advantage of the PAPR of digitized
RF samples of the received signals [120]. The PAPR distribution is used in the
wireless domain to depict a specific class of signal’s footprint such that if
the inferred output of an input signal is categorized to a particular class
and its PAPR statistic contradicts with high level of confidence, further
analysis and assessment should be carried out to reach a true conclusion. To
determine if an adversarial example is present in a wireless communication
environment, the Kolmogorov-Smirnov (KS) two-sample test (performed by
collecting a sample sized input and then computing and evaluating the PAPR
distributions for the samples) can be used to ascertain if the PAPR is similar
to the statistic of an adversarial example or a legitimate example [120].

  * •

Use the softmax outputs of the DNN classifier: This is also a KS statistical
test for adversarial example detection. The softmax output of the DNN
classifier is used to determine if adversarial examples have brought about a
change in distribution using the statistics of the final layer of a neural
network. Typically, the statistical size of the input data is a good
determination of the quality of the statistical test. The effectiveness of the
method depends on the type of waveform and propagation channel, as discussed
in [120].

  * •

Statistical methods to detect adversarial triggers: Adversarial triggers such
as Trojans (or backdoors) inserted to training data can be detected by
statistical methods such as clustering and median absolute deviation (MAD)
algorithm. In particular, the MAD algorithm computes the median of the
absolute deviations from the data’s median, namely
$\text{median}(|x_{i}-\hat{X}|)$, where $\hat{X}=\text{median}(X)$, to
discover outliers, and is shown in [69] to be effective against Trojan attacks
on wireless signal classifiers.

###  VII-C Randomizing the DL Algorithm by Adding Small Variations to
Classifier Outputs

This approach adds slight variations to the output of the DNN classifier and
deliberately performs some (typically a small number of) erroneous actions so
as to deceive the adversary and defend against exploratory attacks. For
example, the transmitter can intentionally utilize a busy channel for
transmitting or skip a transmission opportunity in an idle channel such that
the adversary that observes the spectrum cannot train a reliable surrogate
model. This defense mechanism has been applied to protect wireless signal
classifiers against inference attacks (before launching jamming attacks) in
[117] as well as against evasion and causative attacks in [40]. Note that
network uncertainty (e.g., due to dynamic channel and traffic effects) and
randomization or obfuscation-based defense methods have also been leveraged
before against traditional network attacks such as jamming and traffic
inference to confuse the adversaries [132, 103, 104, 157]. Defense against AML
attacks on reinforcement learning follows a similar diversification approach
to protect wireless communications [76, 79]. The defender may alternate among
different channel access decisions (derived through proportional-integral-
derivative (PID) controller and imitation learning) to increase the
uncertainty at the adversary. Another defense approach is to adopt orthogonal
policies in reinforcement learning to prevent the adversary from quickly
switching between its imitated policies. On top of these mitigation policies,
it is also viable to detect the adversaries by monitoring changes in rewards
and distinguishing the spectrum environment changes from adversarial jamming
attacks [76, 79]. Table VII reviews the defense strategies that have been used
by previous works in the wireless domain.

TABLE VII: Summary of defense strategies against AML attacks in wireless communications. Paper | Defense Strategy
---|---
[57, 58, 117, 40, 67] |  Deliberately makes a small number of incorrect transmit actions so as to fool the adversary and prevent it from building a reliable classifier, e.g., transmit in a busy channel.
[31, 36] |  Slightly modifies the transmitted signal; enough to fool an intruder and allow for correction using the error correction modules of the receiver.
[64] |  Uses an autoencoder for the DL classifier to remove non-salient features thereby removing perturbations.
[69] |  Uses the MAD algorithm and two-step technique of dimensionality reduction and clustering via t-distributed stochastic neighbor embedding (t-SNE) to detect Trojan triggers.
[120, 68, 73] |  Applies KS Statistics that uses the peak-to-average power ratio or the softmax output of ML model for distribution shift detection.
[28] |  Applies randomized smoothing in training time to increase the robustness of classifiers against adversarial attacks in test time and ensures statistical significance of classification results in the presence of adversarial attacks in test time.
[76, 79] |  Applies a diversification approach by alternating the reinforcement learning algorithm among different channel access decisions (via PID controller and imitation learning) and adopting orthogonal policies to restrict the adversary’s response.
[67] |  Adding a constrained perturbation to the signal and ensuring that the signal remains decodable by the oblivious legitimate receiver and minimizing detection accuracy at the intruder.
[73] |  Pre-training the DL classifier using an autoencoder which is expected to filter out non-salient features that may slightly lower the accuracy of classification and makes the classifier more robust to adversarial or environmental corruption.
[118] |  Introduced influence-limiting defense by using decision-flipping influence which indicates the probability of finding a malicious input that modifies the decision output given the unperturbed data by changing part of the unperturbed signal.

##  VIII Future Outlook of Adversarial Machine Learning in Wireless
Communications

As ML/DL becomes the core of current and emerging communications systems such
as 5G and 6G, ML/DL itself becomes susceptible to adversarial effects. To
build on the promise of intelligent operations and efficient resource
management with the application of ML in dynamic wireless environments, it
becomes imperative to develop ML models that are secure, resilient and robust
to attacks by taking into account the unique properties of wireless
communications. An AML attack model that is carefully characterized by
considering wireless properties is expected to be the foundation of ML driven
wireless security research and development for the existing and emerging
communication systems. In this section, we discuss some important ideas that
can help in realizing the promise of robust models that are effective even in
the presence of adversaries.

  * •

Wireless communication dataset: The limitation of standardized real-world
datasets that adequately represent real life scenarios in the wireless
communication domain is a challenge that needs to be solved. Compared to other
domains such as CV and NLP, there are just a handful of publicly available
ML/DL datasets in the wireless domain to work with such as [4, 158, 159, 160,
161, 162]. Typically, these datasets do not include adversarial effects
recorded. It is important for the research community to develop more publicly
available datasets, representing different scenarios including not only
variations in channel, interference type and waveform, but also AML attacks.
This approach will help with benchmarking solutions for robust development and
evaluation of the DL models for wireless versions of AML attacks. From
literature reviewed, a large ratio of the work on AML in wireless
communication is in the area of modulation classification. This is largely
because modulation classification datasets (e.g. [4, 158]) are the seemingly
standardized datasets for Ml/DL studies in the wireless communications domain.
Although more recent datasets are available such as [159, 160, 161, 162] as
well as testbed efforts such as [163] expand datasets, it is critical to
implement the AML attacks and corresponding defenses with embedded radio
platforms to assess the real channel and radio hardware effects.

  * •

Robust features: The need for robust features in developing ML/DL models in
wireless communication is also an important consideration for future systems.
[164] laid the analytical foundation and showed via experimental data that
adversarial attacks are very effective because most existing ML/DL models are
not developed using robust features. Although the discussion of feature
engineering and identifying robust features are on-going especially in CV and
NLP domains, new techniques must be developed to identify the most important
features which are robust to the effect of adversarial attacks in wireless
communication systems. An important research direction for AML in wireless
communications is to identify robust features in datasets, disentangle non-
robust features from robust features, train models that generalize well, and
are robust to adversarial attacks in wireless communications.

  * •

Certifiable defense: The design and development of certifiable defense
mechanisms for AML is another important component of the work that needs to be
achieved to realize the full potential of ML/DL in wireless communications
domain. Various defense mechanisms have been proposed, including adversarial
training and use of statistical methods but most of these defenses become
ineffective with more powerful adversaries. This has led to the need for
certifiable defense mechanisms, where prediction on the test data can be
verified to be a constant within a small region around the input data.
Although there are some efforts that present certifiable defense using
randomized smoothing [28] with reasonable performance, more rigorous
evaluation of such approaches as well as development of other methods are
needed to provide security guarantees for ML models used in wireless
communications.

Although important areas of development to ensure good performance from robust
ML/DL models have been discussed in this section, for the safe adoption of
ML/DL-based models into future communication systems, there is also a need to
move from black-box models to interpretable models. Such paradigm as seen
today includes physics-guided ML [165], where the fundamental laws of physics
is infused into the learning process to improve generalizability and
explainability, and ensure robustness to attacks, as needed in wireless
communication systems.

##  IX Conclusion

The recent trends observed in this survey show that the AML attacks can
effectively disrupt wireless communications. Compared to the traditional
jamming attacks or the addition of noise/interference, adversarial attacks are
more difficult to detect operating with small spectrum footprint, more energy
efficient, and overall more effective. Furthermore, as wireless communications
evolves into smarter systems as in 5G and 6G, the AML attacks also become
smarter. The attacks have evolved from adversaries that send jamming signals
randomly to overwhelm a communication system to those that develop surrogate
models to know the internal workings of the original model and send out small
perturbation signals to fool the ML/DL models and disrupt wireless
communications.

The goal of AML is to support safe adoption of ML/DL solutions to the emerging
applications in the presence of adversaries. Wireless communications strongly
benefit from ML/DL applications in terms of learning from spectrum data and
solving complex tasks. Various studies reviewed in this paper have established
that AML attacks on the ML/DL-driven wireless systems are effective and pose
serious threats leading to major performance degradation. This review attempts
to present the current state of the art in the area of AML attacks specific to
the wireless communications domain and help the research community identify
the research accomplishments as we forge ahead to investigate methods by which
efficient data driven intelligent systems can be built while taking into
account the unique nature of wireless communications, robustness, resilience,
and security of ML/DL based methods. We summarized the latest research efforts
by reviewing the attack types, adversarial example generation, and defense
strategies, established a taxonomy across different attack categories, and
discussed the future outlook to AML in wireless communications. This is an
emerging research area and there is an urgent need for further investigations
to quantify the impact of AML attacks and detect and mitigate them to enable
the safe adoption of the promising ML/DL solutions for wireless
communications.

## References

  * [1] T. Erpek, T. J. O’Shea, Y. E. Sagduyu, Y. Shi, and T. C. Clancy, “Deep learning for wireless communications,” in _Development and Analysis of Deep Learning Architectures_. Springer, 2020, pp. 223–266.
  * [2] O. Simeone, “A very brief introduction to machine learning with applications to communication systems,” _IEEE Transactions on Cognitive Communications and Networking_ , vol. 4, no. 4, pp. 648–664, 2018.
  * [3] L. J. Wong, W. H. Clark IV, B. Flowers, R. M. Buehrer, A. J. Michaels, and W. C. Headley, “The RFML ecosystem: A look at the unique challenges of applying deep learning to radio frequency applications,” _arXiv preprint arXiv:2010.00432_ , 2020.
  * [4] T. J. O’Shea, J. Corgan, and T. C. Clancy, “Convolutional radio modulation recognition networks,” in _Engineering Applications of Neural Networks_ , C. Jayne and L. Iliadis, Eds. Cham: Springer International Publishing, 2016, pp. 213–226.
  * [5] F. Liang, C. Shen, and F. Wu, “An iterative BP-CNN architecture for channel decoding,” _IEEE Journal of Selected Topics in Signal Processing_ , vol. 12, no. 1, pp. 144–159, 2018.
  * [6] H. Jafari, O. Omotere, D. Adesina, H. Wu, and L. Qian, “IoT devices fingerprinting using deep learning,” in _IEEE Military Communications Conference (MILCOM)_ , 2018.
  * [7] T. Jian, B. C. Rendon, E. Ojuba, N. Soltani, Z. Wang, K. Sankhe, A. Gritsenko, J. Dy, K. Chowdhury, and S. Ioannidis, “Deep learning for RF fingerprinting: A massive experimental study,” _IEEE Internet of Things Magazine_ , vol. 3, no. 1, pp. 50–57, 2020.
  * [8] K. Merchant, S. Revay, G. Stantchev, and B. Nousain, “Deep learning for RF device fingerprinting in cognitive communication networks,” _IEEE Journal of Selected Topics in Signal Processing_ , vol. 12, no. 1, pp. 160–167, 2018.
  * [9] K. Davaslioglu, S. Soltani, T. Erpek, and Y. E. Sagduyu, “DeepWiFi: Cognitive WiFi with deep learning,” _IEEE Transactions on Mobile Computing_ , 2019.
  * [10] O. Omotere, J. Fuller, L. Qian, and Z. Han, “Spectrum occupancy prediction in coexisting wireless systems using deep learning,” in _IEEE Vehicular Technology Conference (VTC-Fall)_ , 2018.
  * [11] H. Sun, X. Chen, Q. Shi, M. Hong, X. Fu, and N. D. Sidiropoulos, “Learning to optimize: Training deep neural networks for wireless resource management,” in _IEEE International Workshop on Signal Processing Advances in Wireless Communications (SPAWC)_ , 2017.
  * [12] T. S. Cousik, V. K. Shah, J. H. Reed, T. Erpek, and Y. E. Sagduyu, “Fast initial access with deep learning for beam prediction in 5G mmwave networks,” _arXiv preprint arXiv:2006.12653_ , 2020.
  * [13] N. Abuzainab, M. Alrabeiah, A. Alkhateeb, and Y. E. Sagduyu, “Deep learning for thz drones with flying intelligent surfaces: Beam and handoff prediction,” in _IEEE International Conference on Communications Workshops (ICC Workshops)_ , 2021.
  * [14] C. Szegedy, W. Zaremba, I. Sutskever, J. B. Estrach, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” in _International Conference on Learning Representations (ICLR)_ , 2014.
  * [15] Y. Vorobeychik and M. Kantarcioglu, “Adversarial machine learning,” _Synthesis Lectures on Artificial Intelligence and Machine Learning_ , vol. 12, no. 3, pp. 1–169, December 2017.
  * [16] Y. Shi, Y. E. Sagduyu, K. Davaslioglu, and R. Levy, “Vulnerability detection and analysis in adversarial deep learning,” in _Guide to Vulnerability Analysis for Computer Networks and Systems_. Springer, 2018, pp. 211–234.
  * [17] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” _CoRR_ , vol. abs/1412.6572, 2015.
  * [18] N. Akhtar and A. Mian, “Threat of adversarial attacks on deep learning in computer vision: A survey,” _IEEE Access_ , vol. 6, pp. 14 410–14 430, 2018.
  * [19] H. Xu, Y. Ma, H. Liu, D. Deb, H. Liu, J. Tang, and A. Jain, “Adversarial attacks and defenses in images, graphs and text: A review,” _International Journal of Automation and Computing_ , vol. 17, pp. 151–178, 2020.
  * [20] N. H. Mahmood, H. Alves, O. A. López, M. Shehab, D. P. M. Osorio, and M. Latva-Aho, “Six key features of machine type communication in 6g,” in _2020 2nd 6G Wireless Summit (6G SUMMIT)_ , 2020, pp. 1–5.
  * [21] M. G. Kibria, K. Nguyen, G. P. Villardi, O. Zhao, K. Ishizu, and F. Kojima, “Big data analytics, machine learning, and artificial intelligence in next-generation wireless networks,” _IEEE Access_ , vol. 6, pp. 32 328–32 338, 2018.
  * [22] G. Wikström, J. Peisa, P. Rugeland, N. Johansson, S. Parkvall, M. Girnyk, G. Mildh, and I. L. Da Silva, “Challenges and technologies for 6g,” in _2020 2nd 6G Wireless Summit (6G SUMMIT)_ , 2020, pp. 1–5.
  * [23] L. Zhang, Y.-C. Liang, and D. Niyato, “6G visions: Mobile ultra-broadband, super internet-of-things, and artificial intelligence,” _China Communications_ , vol. 16, no. 8, pp. 1–14, 2019.
  * [24] X. Liao, L. Ding, and Y. Wang, “Secure machine learning, a brief overview,” in _2011 Fifth International Conference on Secure Software Integration and Reliability Improvement - Companion_ , 2011, pp. 26–29.
  * [25] M. Barreno, B. Nelson, R. Sears, A. D. Joseph, and J. D. Tygar, “Can machine learning be secure?” in _ACM Symposium on Information, computer and communications security_ , 2006, pp. 16–25.
  * [26] F. O. Olowononi, D. B. Rawat, and C. Liu, “Resilient machine learning for networked cyber physical systems: A survey for machine learning security to securing machine learning for cps,” _IEEE Communications Surveys Tutorials_ , vol. 23, no. 1, pp. 524–552, 2021.
  * [27] A. Qayyum, M. Usama, J. Qadir, and A. Al-Fuqaha, “Securing connected autonomous vehicles: Challenges posed by adversarial machine learning and the way forward,” _IEEE Communications Surveys Tutorials_ , vol. 22, no. 2, pp. 998–1026, 2020.
  * [28] B. Kim, Y. E. Sagduyu, K. Davaslioglu, T. Erpek, and S. Ulukus, “Channel-aware adversarial attacks against deep learning-based wireless signal classifiers,” _arXiv preprint arXiv:2005.05321_ , 2020.
  * [29] ——, “Over-the-air adversarial attacks on deep learning based modulation classifier over wireless channels,” in _2020 54th Annual Conference on Information Sciences and Systems (CISS)_ , 2020.
  * [30] B. Kim, Y. E. Sagduyu, T. Erpek, K. Davaslioglu, and S. Ulukus, “Adversarial attacks with multiple antennas against deep learning-based modulation classifiers,” in _IEEE Globecom Workshops_ , 2020.
  * [31] M. Z. Hameed, A. Gyorgy, and D. Gunduz, “The best defense is a good offense: Adversarial attacks to avoid modulation detection,” _IEEE Transactions on Information Forensics and Security_ , vol. 16, pp. 1074–1087, 2020.
  * [32] F. Restuccia, S. D’Oro, A. Al-Shawabka, B. C. Rendon, K. Chowdhury, S. Ioannidis, and T. Melodia, “Generalized wireless adversarial deep learning,” in _ACM Workshop on Wireless Security and Machine Learning (WiseML)_ , 2020.
  * [33] B. Kim, Y. E. Sagduyu, T. Erpek, K. Davaslioglu, and S. Ulukus, “Channel effects on surrogate models of adversarial attacks against wireless signal classifiers,” in _IEEE International Conference on Communications_ , 2021\.
  * [34] R. Sahay, C. G. Brinton, and D. J. Love, “Frequency-based automated modulation classification in the presence of adversaries,” _arXiv preprint arXiv:2011.01132_ , 2020.
  * [35] B. Kim, Y. E. Sagduyu, K. Davaslioglu, T. Erpek, and S. Ulukus, “How to make 5G communications” invisible”: Adversarial machine learning for wireless privacy,” _arXiv preprint arXiv:2005.07675_ , 2020.
  * [36] M. Z. Hameed, A. Gyorgy, and D. Gunduz, “Communication without interception: Defense against modulation detection,” in _IEEE Global Conference on Signal and Information Processing (GlobalSIP)_ , 2019.
  * [37] A. Berian, K. Staab, N. Teku, G. Ditzler, T. Bose, and R. Tandon, “Adversarial filters for secure modulation classification,” _arXiv preprint arXiv:2008.06785_ , 2020.
  * [38] M. DelVecchio, V. Arndorfer, and W. C. Headley, “Investigating a spectral deception loss metric for training machine learning-based evasion attacks,” _arXiv preprint arXiv:2005.13124_ , 2020.
  * [39] M. DelVecchio, B. Flowers, and W. C. Headley, “Effects of forward error correction on communications aware evasion attacks,” _arXiv preprint arXiv:2005.13123_ , 2020.
  * [40] Y. E. Sagduyu, Y. Shi, and T. Erpek, “Adversarial deep learning for over-the-air spectrum poisoning attacks,” _IEEE Transactions on Mobile Computing_ , vol. 20, no. 2, pp. 306–319, 2021.
  * [41] Y. Lin, H. Zhao, Y. Tu, S. Mao, and Z. Dou, “Threats of adversarial attacks in DNN-based modulation recognition,” in _IEEE Conference on Computer Communications (INFOCOM)_ , 2020.
  * [42] Y. E. Sagduyu, R. A. Berry, and D. Guo, “Throughput and stability for relay-assisted wireless broadcast with network coding,” _IEEE Journal on Selected Areas in Communications_ , vol. 31, no. 8, pp. 1506–1516, 2013.
  * [43] Y. E. Sagduyu, Y. Shi, T. Erpek, W. Headley, B. Flowers, G. Stantchev, and Z. Lu, “When wireless security meets machine learning: Motivation, challenges, and research directions,” _arXiv preprint arXiv:2001.08883_ , 2020.
  * [44] D. Roy, T. Mukherjee, and M. Chatterjee, “Machine learning in adversarial RF environments,” _IEEE Communications Magazine_ , vol. 57, no. 5, pp. 82–87, 2019.
  * [45] D. Adesina, J. Bassey, and L. Qian, “Robust deep radio frequency spectrum learning for future wireless communications systems,” _IEEE Access_ , vol. 8, pp. 148 528–148 540, 2020.
  * [46] D. Adesina., J. Bassey, and L. Qian, “Practical radio frequency learning for future wireless communication systems,” in _IEEE Military Communications Conference (MILCOM)_ , 2019.
  * [47] Y. Shi, K. Davaslioglu, Y. E. Sagduyu, W. C. Headley, M. Fowler, and G. Green, “Deep learning for RF signal classification in unknown and dynamic spectrum environments,” in _IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN)_ , 2019.
  * [48] Y. Sun, M. Peng, Y. Zhou, Y. Huang, and S. Mao, “Application of machine learning in wireless networks: Key techniques and open issues,” _IEEE Communications Surveys Tutorials_ , vol. 21, no. 4, pp. 3072–3108, 2019.
  * [49] L. Dai, R. Jiao, F. Adachi, H. V. Poor, and L. Hanzo, “Deep learning for wireless communications: An emerging interdisciplinary paradigm,” _IEEE Wireless Communications_ , vol. 27, no. 4, pp. 133–139, 2020.
  * [50] M. Kulin, T. Kazaz, E. De Poorter, and I. Moerman, “A survey on machine learning-based performance improvement of wireless networks: Phy, mac and network layer,” _Electronics_ , vol. 10, no. 3, 2021.
  * [51] P. V. Klaine, M. A. Imran, O. Onireti, and R. D. Souza, “A survey of machine learning techniques applied to self-organizing cellular networks,” _IEEE Communications Surveys Tutorials_ , vol. 19, no. 4, pp. 2392–2431, 2017\.
  * [52] D. C. Nguyen, P. Cheng, M. Ding, D. Lopez-Perez, P. N. Pathirana, J. Li, A. Seneviratne, Y. Li, and H. V. Poor, “Enabling ai in future wireless networks: A data life cycle perspective,” _IEEE Communications Surveys Tutorials_ , vol. 23, no. 1, pp. 553–595, 2021.
  * [53] J. Jagannath, N. Polosky, A. Jagannath, F. Restuccia, and T. Melodia, “Machine learning for wireless communications in the internet of things: A comprehensive survey,” _Ad Hoc Networks_ , vol. 93, 2019.
  * [54] M. Chen, U. Challita, W. Saad, C. Yin, and M. Debbah, “Artificial neural networks-based machine learning for wireless networks: A tutorial,” _IEEE Communications Surveys Tutorials_ , vol. 21, no. 4, pp. 3039–3071, 2019\.
  * [55] C. Jiang, H. Zhang, Y. Ren, Z. Han, K.-C. Chen, and L. Hanzo, “Machine learning paradigms for next-generation wireless networks,” _IEEE Wireless Communications_ , vol. 24, no. 2, pp. 98–105, 2017.
  * [56] G. Y. Li, W. Saad, A. Ozgur, P. Kairouz, Z. Qin, J. Hoydis, Z. Han, D. Gunduz, and J. Elmirghani, “Series editorial: The second issue of the series on machine learning in communications and networks,” _IEEE Journal on Selected Areas in Communications_ , vol. 39, no. 7, pp. 1855–1857, 2021.
  * [57] Y. E. Sagduyu, Y. Shi, and T. Erpek, “IoT network security from the perspective of adversarial deep learning,” in _IEEE International Conference on Sensing, Communication, and Networking (SECON)_ , 2019.
  * [58] Y. Shi, Y. E. Sagduyu, T. Erpek, K. Davaslioglu, Z. Lu, and J. H. Li, “Adversarial deep learning for cognitive radio security: Jamming attack and defense strategies,” in _IEEE International Conference on Communications Workshops (ICC Workshops)_ , 2018.
  * [59] Y. Shi, T. Erpek, Y. E. Sagduyu, and J. H. Li, “Spectrum data poisoning with adversarial deep learning,” in _IEEE Military Communications Conference (MILCOM)_ , 2018.
  * [60] M. Sadeghi. and E. G. Larsson, “Physical adversarial attacks against end-to-end autoencoder communication systems,” _IEEE Communications Letters_ , vol. 23, no. 5, pp. 847–850, 2019.
  * [61] Y. Shi, K. Davaslioglu, and Y. E. Sagduyu, “Over-the-air membership inference attacks as privacy threats for deep learning-based wireless signal classifiers,” in _ACM Workshop on Wireless Security and Machine Learning (WiseML)_ , 2020.
  * [62] Y. Shi and Y. E. Sagduyu, “Membership inference attack and defense for wireless signal classifiers with deep learning,” _arXiv preprint arXiv:2107.12173_ , 2021.
  * [63] M. Sadeghi and E. G. Larsson, “Adversarial attacks on deep-learning based radio signal classification,” _IEEE Wireless Communications Letters_ , vol. 8, no. 1, pp. 213–216, 2019.
  * [64] S. Kokalj-Filipovic, R. Miller, N. Chang, and C. L. Lau, “Mitigation of adversarial examples in RF deep classifiers utilizing autoencoder pre-training,” in _International Conference on Military Communications and Information Systems (ICMCIS)_ , 2019.
  * [65] S. Bair, M. DelVecchio, B. Flowers, A. J. Michaels, and W. C. Headley, “On the limitations of targeted adversarial evasion attacks against deep learning enabled modulation recognition,” in _ACM Workshop on Wireless Security and Machine Learning (WiseML)_ , 2019.
  * [66] D. Ke, Z. Huang, X. Wang, and L. Sun, “Application of adversarial examples in communication modulation classification,” in _International Conference on Data Mining Workshops (ICDMW)_ , 2019.
  * [67] M. Z. Hameed, A. György, and D. Gündüz, “Communication without interception: Defense against modulation detection,” _IEEE Global Conference on Signal and Information Processing (GlobalSIP)_ , 2019.
  * [68] S. Kokalj-Filipovic and R. Miller, “Adversarial examples in RF learning: detection of the attack and its physical robustness,” _arXiv preprint arXiv:1902.06044_ , 2019.
  * [69] K. Davaslioglu and Y. E. Sagduyu, “Trojan attacks on wireless signal classification with adversarial machine learning,” in _IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN)_ , 2019.
  * [70] Q. Liu, J. Guo, C. Wen, and S. Jin, “Adversarial attack on DL-based massive MIMO CSI feedback,” _Journal of Communications and Networks_ , vol. 22, no. 3, pp. 230–235, 2020.
  * [71] B. Flowers, R. M. Buehrer, and W. C. Headley, “Evaluating adversarial evasion attacks in the context of wireless communications,” _IEEE Transactions on Information Forensics and Security_ , vol. 15, pp. 1102–1113, 2020\.
  * [72] F. Restuccia, S. D’Oro, A. Al-Shawabka, B. C. Rendon, K. Chowdhury, S. Ioannidis, and T. Melodia, “Hacking the waveform: Generalized wireless adversarial deep learning,” _arXiv preprint arXiv:2005.02270_ , 2020.
  * [73] S. Kokalj-Filipovic, R. Miller, and J. Morman, “Targeted adversarial examples against RF deep classifiers,” in _ACM Workshop on Wireless Security and Machine Learning (WiseML)_ , 2019.
  * [74] S. Karunaratne, E. Krijestorac, and D. Cabric, “Penetrating RF fingerprinting-based authentication with a generative adversarial attack,” _arXiv preprint arXiv:2011.01538_ , 2020.
  * [75] C. Zhong, F. Wang, M. C. Gursoy, and S. Velipasalar, “Adversarial jamming attacks on deep reinforcement learning based dynamic multichannel access,” in _IEEE Wireless Communications and Networking Conference (WCNC)_ , 2020\.
  * [76] F. Wang, C. Zhong, M. C. Gursoy, and S. Velipasalar, “Adversarial jamming attacks and defense strategies via adaptive deep reinforcement learning,” 2020\.
  * [77] Y. Shi, Y. E. Sagduyu, T. Erpek, and M. C. Gursoy, “How to attack and defend 5G radio access network slicing with reinforcement learning,” _arXiv preprint arXiv:2101.05768_ , 2021.
  * [78] Y. Shi and Y. E. Sagduyu, “Adversarial machine learning for flooding attacks on 5g radio access network slicing,” in _IEEE International Conference on Communications Workshops (ICC Workshops)_ , 2021.
  * [79] F. Wang, C. Zhong, M. C. Gursoy, and S. Velipasalar, “Defense strategies against adversarial jamming attacks via deep reinforcement learning,” in _Conference on Information Sciences and Systems (CISS)_ , 2020.
  * [80] J. Deogirikar and A. Vidhate, “Security attacks in iot: A survey,” in _2017 International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC)_ , 2017, pp. 32–37.
  * [81] F. Shahzad, M. Pasha, and A. Ahmad, “A survey of active attacks on wireless sensor networks and their countermeasures,” 2017.
  * [82] S. Jaitly, H. Malhotra, and B. Bhushan, “Security vulnerabilities and countermeasures against jamming attacks in wireless sensor networks: A survey,” in _2017 International Conference on Computer, Communications and Electronics (Comptelix)_ , 2017, pp. 559–564.
  * [83] S. Vadlamani, B. Eksioglu, H. Medal, and A. Nandi, “Jamming attacks on wireless networks: A taxonomic survey,” _International Journal of Production Economics_ , vol. 172, pp. 76–94, 2016.
  * [84] Y. Wu, A. Khisti, C. Xiao, G. Caire, K.-K. Wong, and X. Gao, “A survey of physical layer security techniques for 5g wireless networks and challenges ahead,” _IEEE Journal on Selected Areas in Communications_ , vol. 36, no. 4, pp. 679–695, 2018.
  * [85] I. Stellios, P. Kotzanikolaou, M. Psarakis, C. Alcaraz, and J. Lopez, “A survey of iot-enabled cyberattacks: Assessing attack paths to critical infrastructures and services,” _IEEE Communications Surveys Tutorials_ , vol. 20, no. 4, pp. 3453–3495, 2018.
  * [86] Y. Zou, J. Zhu, X. Wang, and L. Hanzo, “A survey on wireless security: Technical challenges, recent advances, and future trends,” _Proceedings of the IEEE_ , vol. 104, no. 9, pp. 1727–1765, 2016.
  * [87] M. Arif, G. Wang, M. Zakirul Alam Bhuiyan, T. Wang, and J. Chen, “A survey on security attacks in vanets: Communication, applications and challenges,” _Vehicular Communications_ , vol. 19, 2019.
  * [88] P. Dewal, G. S. Narula, V. Jain, and A. Baliyan, “Security attacks in wireless sensor networks: A survey,” in _Cyber Security_ , M. U. Bokhari, N. Agrawal, and D. Saini, Eds. Singapore: Springer Singapore, 2018, pp. 47–58.
  * [89] M. Parashar, A. Poonia, and K. Satish, “A survey of attacks and their mitigations in software defined networks,” in _2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT)_ , 2019, pp. 1–8.
  * [90] S. Bhambri, S. Muku, A. Tulasi, and A. B. Buduru, “A survey of black-box adversarial attacks on computer vision models,” 2020.
  * [91] X. Yuan, P. He, Q. Zhu, and X. Li, “Adversarial examples: Attacks and defenses for deep learning,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 30, no. 9, pp. 2805–2824, 2019.
  * [92] I. Tomić and J. A. McCann, “A survey of potential security issues in existing wireless sensor network protocols,” _IEEE Internet of Things Journal_ , vol. 4, no. 6, pp. 1910–1923, 2017.
  * [93] B. Biggio and F. Roli, “Wild patterns: Ten years after the rise of adversarial machine learning,” _Pattern Recognition_ , vol. 84, p. 317–331, Dec 2018\.
  * [94] L. Huang, A. D. Joseph, B. Nelson, B. I. Rubinstein, and J. D. Tygar, “Adversarial machine learning,” in _Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence_ , ser. AISec ’11. New York, NY, USA: Association for Computing Machinery, 2011, p. 43–58.
  * [95] A. Chakraborty, M. Alam, V. Dey, A. Chattopadhyay, and D. Mukhopadhyay, “Adversarial attacks and defences: A survey,” 2018.
  * [96] E. Nowroozi, A. Dehghantanha, R. M. Parizi, and K.-K. R. Choo, “A survey of machine learning techniques in adversarial image forensics,” _Computers and Security_ , vol. 100, 2021.
  * [97] G. R. Machado, E. Silva, and R. R. Goldschmidt, “Adversarial machine learning in image classification: A survey towards the defender’s perspective,” 2020.
  * [98] A. Serban, E. Poll, and J. Visser, “Adversarial examples on object recognition: A comprehensive survey,” _ACM Comput. Surv._ , vol. 53, no. 3, Jun. 2020.
  * [99] L. Sun, Y. Dou, C. Yang, J. Wang, P. S. Yu, L. He, and B. Li, “Adversarial attack and defense on graph data: A survey,” 2020.
  * [100] Y. Zou, J. Zhu, L. Yang, Y. Liang, and Y. Yao, “Securing physical-layer communications for cognitive radio networks,” _IEEE Communications Magazine_ , vol. 53, no. 9, pp. 48–54, 2015.
  * [101] Y. Zou, J. Zhu, X. Wang, and L. Hanzo, “A survey on wireless security: Technical challenges, recent advances, and future trends,” _Proceedings of the IEEE_ , vol. 104, no. 9, pp. 1727–1765, 2016.
  * [102] Y. E. Sagduyu, “Securing cognitive radio networks with dynamic trust against spectrum sensing data falsification,” in _IEEE Military Communications Conference_ , 2014.
  * [103] Y. E. Sagduyu, R. A. Berry, and A. Ephremides, “Jamming games in wireless networks with incomplete information,” _IEEE Communications Magazine_ , vol. 49, no. 8, pp. 112–118, 2011.
  * [104] ——, “Wireless jamming attacks under dynamic traffic uncertainty,” in _International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt)_ , 2010.
  * [105] S. Rajendran, W. Meert, V. Lenders, and S. Pollin, “Saife: Unsupervised wireless spectrum anomaly detection with interpretable features,” in _IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN)_ , 2018.
  * [106] N. Van Huynh, D. N. Nguyen, D. T. Hoang, and E. Dutkiewicz, “Jam me if you can: Defeating jammer with deep dueling neural network architecture and ambient backscattering augmented communications,” _IEEE Journal on Selected Areas in Communications_ , vol. 37, no. 11, pp. 2603–2620, 2019.
  * [107] N. Abuzainab, T. Erpek, K. Davaslioglu, Y. E. Sagduyu, Y. Shi, S. J. Mackey, M. Patel, F. Panettieri, M. A. Qureshi, V. Isler _et al._ , “QoS and jamming-aware wireless networking using deep reinforcement learning,” in _IEEE Military Communications Conference (MILCOM)_ , 2019.
  * [108] Y. Zhou, M. Kantarcioglu, and B. Xi, “A survey of game theoretic approach for adversarial machine learning,” _Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery_ , vol. 9, no. 3, p. e1259, 2019.
  * [109] X. Wang, J. Li, X. Kuang, Y. an Tan, and J. Li, “The security of machine learning in an adversarial setting: A survey,” _Journal of Parallel and Distributed Computing_ , vol. 130, pp. 12–23, 2019.
  * [110] G. Li, P. Zhu, J. Li, Z. Yang, N. Cao, and Z. Chen, “Security matters: A survey on adversarial machine learning,” _arXiv preprint arXiv:1810.07339_ , 2018.
  * [111] M. Ozdag, “Adversarial attacks and defenses against deep neural networks: A survey,” _Procedia Computer Science_ , vol. 140, pp. 152–161, 2018, cyber Physical Systems and Deep Learning Chicago, Illinois November 5-7, 2018\.
  * [112] Q. Liu, P. Li, W. Zhao, W. Cai, S. Yu, and V. C. M. Leung, “A survey on security threats and defensive techniques of machine learning: A data driven view,” _IEEE Access_ , vol. 6, pp. 12 103–12 117, 2018.
  * [113] N. Pitropakis, E. Panaousis, T. Giannetsos, E. Anastasiadis, and G. Loukas, “A taxonomy and survey of attacks against machine learning,” _Computer Science Review_ , vol. 34, p. 100199, 2019.
  * [114] S. H. Silva and P. Najafirad, “Opportunities and challenges in deep learning adversarial robustness: A survey,” 2020.
  * [115] R. R. Wiyatno, A. Xu, O. Dia, and A. de Berker, “Adversarial examples in modern machine learning: A review,” 2019.
  * [116] I. Goodfellow, Y. Bengio, and A. Courville, _Deep Learning_. MIT Press, 2016, http://www.deeplearningbook.org.
  * [117] T. Erpek, Y. E. Sagduyu, and Y. Shi, “Deep learning for launching and mitigating wireless jamming attacks,” _IEEE Transactions on Cognitive Communications and Networking_ , vol. 5, no. 1, pp. 2–14, 2019.
  * [118] Z. Luo, S. Zhao, Z. Lu, J. Xu, and Y. E. Sagduyu, “When attackers meet AI: Learning-empowered attacks in cooperative spectrum sensing,” _IEEE Transactions on Mobile Computing_ , 2020.
  * [119] Z. Luo, S. Zhao, Z. Lu, Y. E. Sagduyu, and J. Xu, “Adversarial machine learning based partial-model attack in IoT,” in _ACM Workshop on Wireless Security and Machine Learning (WiseML)_ , 2020.
  * [120] S. Kokalj-Filipovic, R. Miller, and G. Vanhoy, “Adversarial examples in RF deep learning: Detection and physical robustness,” in _IEEE Global Conference on Signal and Information Processing (GlobalSIP)_ , 2019.
  * [121] Y. Shi, T. Erpek, Y. E. Sagduyu, and J. H. Li, “Spectrum data poisoning with adversarial deep learning,” in _IEEE Military Communications Conference (MILCOM)_ , 2018.
  * [122] H. Kwon, Y. Kim, H. Yoon, and D. Choi, “Fooling a neural network in military environments: Random untargeted adversarial example,” in _IEEE Military Communications Conference (MILCOM)_ , 2018.
  * [123] N. Akhtar and A. Mian, “Threat of adversarial attacks on deep learning in computer vision: A survey,” _IEEE Access_ , vol. 6, pp. 14 410–14 430, 2018.
  * [124] N. Papernot, P. McDaniel, A. Sinha, and M. P. Wellman, “Sok: Security and privacy in machine learning,” in _2018 IEEE European Symposium on Security and Privacy (EuroS P)_ , 2018, pp. 399–414.
  * [125] F. Tramèr, F. Zhang, A. Juels, M. K. Reiter, and T. Ristenpart, “Stealing machine learning models via prediction APIs,” in _USENIX Security Symposium_ , 2016.
  * [126] Yi Shi, Y. E. Sagduyu, and A. Grushin, “How to steal a machine learning classifier with deep learning,” in _IEEE International Symposium on Technologies for Homeland Security (HST)_ , 2017.
  * [127] Y. Shi, Y. E. Sagduyu, K. Davaslioglu, and J. H. Li, “Active deep learning attacks under strict rate limitations for online API calls,” in _IEEE International Symposium on Technologies for Homeland Security (HST)_ , 2018.
  * [128] ——, “Generative adversarial networks for black-box API attacks with limited training data,” in _2018 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT)_ , 2018.
  * [129] N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, and A. Swami, “Practical black-box attacks against machine learning,” in _ACM Asia Conference on Computer and Communications Security (AsiaCCS)_ , 2017.
  * [130] B. Manoj, M. Sadeghi, and E. G. Larsson, “Adversarial attacks on deep learning based power allocation in a massive MIMO network,” _arXiv preprint arXiv:2101.12090_ , 2021.
  * [131] B. Kim, Y. E. Sagduyu, T. Erpek, and S. Ulukus, “Adversarial attacks on deep learning based mmwave beam prediction in 5G and beyond,” _arXiv preprint arXiv:2103.13989_ , 2021.
  * [132] W. Xu, W. Trappe, Y. Zhang, and T. Wood, “The feasibility of launching and detecting jamming attacks in wireless networks,” in _ACM International Symposium on Mobile Ad Hoc Networking and Computing_ , 2005, pp. 46–57.
  * [133] B. Biggio, B. Nelson, and P. Laskov, “Poisoning attacks against support vector machines,” in _International Conference on Machine Learning (ICML)_ , 2012\.
  * [134] Y. Shi and Y. E. Sagduyu, “Evasion and causative attacks with adversarial deep learning,” in _IEEE Military Communications Conference (MILCOM)_ , 2017.
  * [135] Z. Luo, S. Zhao, R. Duan, Z. Lu, Y. E. Sagduyu, and J. Xu, “Low-cost influence-limiting defense against adversarial machine learning attacks in cooperative spectrum sensing,” in _ACM Workshop on Wireless Security and Machine Learning (WiseML)_ , 2021.
  * [136] Y. He, G. Meng, K. Chen, X. Hu, and J. He, “Towards security threats of deep learning systems: A survey,” _IEEE Transactions on Software Engineering_ , pp. 1–1, 2020.
  * [137] B. Wang, Y. Yao, S. Shan, H. Li, B. Viswanath, H. Zheng, and B. Y. Zhao, “Neural cleanse: Identifying and mitigating backdoor attacks in neural networks,” in _IEEE Symposium on Security and Privacy (SP)_ , 2019.
  * [138] B. Chen, W. Carvalho, N. Baracaldo, H. Ludwig, B. Edwards, T. Lee, I. Molloy, and B. Srivastava, “Detecting backdoor attacks on deep neural networks by activation clustering,” _arXiv preprint arXiv:1811.03728_ , 2018.
  * [139] A. Kurakin, I. Goodfellow, and S. Bengio, “Adversarial examples in the physical world,” _arXiv preprint arXiv:1607.02533_ , 2016.
  * [140] S.-M. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, and P. Frossard, “Universal adversarial perturbations,” in _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2017.
  * [141] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” _arXiv preprint arXiv:1706.06083_ , 2017.
  * [142] Y. Dong, F. Liao, T. Pang, H. Su, J. Zhu, X. Hu, and J. Li, “Boosting adversarial attacks with momentum,” in _IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2018.
  * [143] O. A. Dobre, A. Abdi, Y. Bar-Ness, and W. Su, “Survey of automatic modulation classification techniques: classical approaches and new trends,” _IET Communications_ , vol. 1, no. 2, pp. 137–156, April 2007.
  * [144] T. J. O’Shea, J. Corgan, and T. C. Clancy, “Convolutional radio modulation recognition networks,” in _Engineering Applications of Neural Networks_ , C. Jayne and L. Iliadis, Eds. Cham: Springer International Publishing, 2016, pp. 213–226.
  * [145] T. J. O’shea and N. West, “Radio machine learning dataset generation with GNU Radio,” in _GNU Radio Conference_ , 2016.
  * [146] M. Z. Hameed. (2020, July) New quality measures for adversarial attacks with applications to secure communication. Online. [Online]. Available: https://spiral.imperial.ac.uk:8443/handle/10044/1/82214
  * [147] T. J. O’Shea, T. Roy, and T. C. Clancy, “Over-the-air deep learning based radio signal classification,” _IEEE Journal of Selected Topics in Signal Processing_ , vol. 12, no. 1, pp. 168–179, 2018.
  * [148] J. Yi and A. E. Gamal, “Gradient-based adversarial deep modulation classification with data-driven subsampling,” 2021.
  * [149] T. J. O’Shea and N. West, “Radio machine learning dataset generation with gnu radio,” _Proceedings of the 6th GNU Radio Conference_ , 2016.
  * [150] A. Bahramali, M. Nasr, A. Houmansadr, D. Goeckel, and D. Towsley, “Robust adversarial attacks against dnn-based wireless communication systems,” 2021.
  * [151] A. Saeed, K. A. Harras, E. Zegura, and M. Ammar, “Local and low-cost white space detection,” in _IEEE International Conference on Distributed Computing Systems (ICDCS)_ , 2017.
  * [152] Y. E. Sagduyu, T. Erpek, and Y. Shi, “Adversarial machine learning for 5G communications security,” _arXiv preprint arXiv:2101.02656_ , 2021.
  * [153] L. Liu, C. Oestges, J. Poutanen, K. Haneda, P. Vainikainen, F. Quitin, F. Tufvesson, and P. D. Doncker, “The COST 2100 MIMO channel model,” _IEEE Wireless Communications_ , vol. 19, no. 6, pp. 92–99, 2012.
  * [154] L. Sanguinetti, A. Zappone, and M. Debbah, “Deep learning power allocation in massive mimo,” 2019.
  * [155] J. M. Cohen, E. Rosenfeld, and J. Z. Kolter, “Certified adversarial robustness via randomized smoothing,” _arXiv preprint arXiv:1902.02918_ , 2019.
  * [156] A. Raghunathan, J. Steinhardt, and P. Liang, “Certified defenses against adversarial examples,” in _International Conference on Learning Representations (ICLR)_ , 2018.
  * [157] T. Hou, Z. Qu, T. Wang, Z. Lu, and Y. Liu, “Proto: Proactive topology obfuscation against adversarial network topology inference,” in _IEEE Conference on Computer Communications_ , 2020.
  * [158] T. J. O’Shea, T. Roy, and T. C. Clancy, “Over-the-air deep learning based radio signal classification,” _IEEE Journal of Selected Topics in Signal Processing_ , vol. 12, no. 1, pp. 168–179, Feb 2018.
  * [159] G. Lab, “Genesys lab ml datasets,” https://genesys-lab.org/mldatasets, 2020.
  * [160] A. Al-shawabka, F. Restuccia, S. D’Oro, and T. Melodia, “Massive-scale I/Q datasets for WiFi radio fingerprinting,” _Computer Networks_ , vol. 182, p. 107566, 2020.
  * [161] A. Alkhateeb, “DeepMIMO: A generic deep learning dataset for millimeter wave and massive MIMO applications,” in _Information Theory and Applications Workshop (ITA)_ , 2019.
  * [162] M. Ezuma, F. Erden, C. K. Anjinappa, O. Ozdemir, and I. Guvenc, “Drone remote controller RF signal dataset,” 2020. [Online]. Available: https://dx.doi.org/10.21227/ss99-8d56
  * [163] J. Breen, A. Buffmire, J. Duerig, K. Dutt, E. Eide, M. Hibler, D. Johnson, S. K. Kasera, E. Lewis, D. Maas, A. Orange, N. Patwari, D. Reading, R. Ricci, D. Schurig, L. B. Stoller, J. Van der Merwe, K. Webb, and G. Wong, “POWDER: Platform for open wireless data-driven experimental research,” in _Proceedings of the 14th International Workshop on Wireless Network Testbeds, Experimental Evaluation and Characterization (WiNTECH)_ , Sep. 2020.
  * [164] A. Ilyas, S. Santurkar, D. Tsipras, L. Engstrom, B. Tran, and A. Madry, “Adversarial examples are not bugs, they are features,” 2019.
  * [165] R. Rai and C. K. Sahu, “Driven by data or derived through physics? a review of hybrid physics guided machine learning techniques with cyber-physical system (cps) focus,” _IEEE Access_ , vol. 8, pp. 71 050–71 073, 2020.