Correlation-Aware Neural Networks for DDoS Attack Detection In IoT
  Systems
We present a comprehensive study on applying machine learning to detect
distributed Denial of service (DDoS) attacks using large-scale Internet of
Things (IoT) systems. While prior works and existing DDoS attacks have largely
focused on individual nodes transmitting packets at a high volume, we
investigate more sophisticated futuristic attacks that use large numbers of IoT
devices and camouflage their attack by having each node transmit at a volume
typical of benign traffic. We introduce new correlation-aware architectures
that take into account the correlation of traffic across IoT nodes, and we also
compare the effectiveness of centralized and distributed detection models. We
extensively analyze the proposed architectures by evaluating five different
neural network models trained on a dataset derived from a 4060-node real-world
IoT system. We observe that long short-term memory (LSTM) and a
transformer-based model, in conjunction with the architectures that use
correlation information of the IoT nodes, provide higher performance (in terms
of F1 score and binary accuracy) than the other models and architectures,
especially when the attacker camouflages itself by following benign traffic
distribution on each transmitting node. For instance, by using the LSTM model,
the distributed correlation-aware architecture gives 81% F1 score for the
attacker that camouflages their attack with benign traffic as compared to 35%
for the architecture that does not use correlation information. We also
investigate the performance of heuristics for selecting a subset of nodes to
share their data for correlation-aware architectures to meet resource
constraints.
Arvin Hekmati, Nishant Jethwa, Eugenio Grippo, Bhaskar Krishnamachari
# Correlation-Aware Neural Networks for DDoS Attack Detection In IoT Systems

###### Abstract

We present a comprehensive study on applying machine learning to detect distributed denial of service (DDoS) attacks using large-scale Internet of Things (IoT) systems. While prior works and existing DDoS attacks have largely focused on individual nodes transmitting packets at a high volume, we investigate more sophisticated futuristic attacks that use large numbers of IoT devices and camouflage their attack by having each node transmit at a volume typical of benign traffic. We introduce new correlation-aware architectures that take into account the correlation of traffic across IoT nodes, and we also compare the effectiveness of centralized and distributed detection models. We extensively analyze the proposed architectures by evaluating five different neural network models trained on a dataset derived from a 4060-node real-world IoT system. We observe that long short-term memory (LSTM) and a transformer-based model, in conjunction with the architectures that use correlation information of the IoT nodes, provide higher performance (in terms of F1 score and binary accuracy) than the other models and architectures, especially when the attacker camouflages itself by following benign traffic distribution on each transmitting node. For instance, by using the LSTM model, the distributed correlation-aware architecture gives 81% F1 score for the attacker that camouflages their attack with benign traffic as compared to 35% for the architecture that does not use correlation information. We also investigate the performance of heuristics for selecting a subset of nodes to share their data for correlation-aware architectures to meet resource constraints.

 IoT DDoS Attacks, datasets, neural networks, machine learning, botnet, Cauchy distribution

## I Introduction

The Internet of things (IoT) has dramatically grown, propelled by the impetuous development of technology [1, 2]. IoT devices are predicted to be more pervasive in our lives as compared to mobile devices. The number of IoT devices (nodes) connected to the Internet is projected to be around 29 billion devices around the world [3]. Along with the growth of the number of IoT devices, the vulnerability and security risks of IoT devices have also been increasing almost at the same pace. Recent studies by HP security research showed a high average number of vulnerabilities by studying 10 of the most popular IoT devices, such as lack of transport encryption, insecure software/irmmware, and cross-site scripting [4, 5]. As a consequence, cybersecurity of IoT devices ought to be developed at an ever-faster rhythm, allowing/accompanying the safe and secure growth of networks [6, 7]. With this goal in mind, this work addresses one of the most dangerous types of attacks involving IoT systems, namely distributed denial of service (DDoS) attacks [8, 9, 10].

In denial of service (DoS) attacks, the attacker tries to disturb the behavior of the victim server in order to block legitimate users' system access by reducing the server's availability. DoS attacks come in many forms, such as user datagram protocol (UDP) flood attacks, where the attacker sends more traffic to the victim server than it is capable of handling [11], synchronize (SYN) flood attacks, where the attacker sends a transmission control protocol (TCP) connection request to the victim server with spoofed source addresses but never send back the acknowledgment [12], etc.

In distributed denial of service (DDoS) attacks, attackers get access to many compromised devices, usually called zombies, in order to perform DoS attacks on the victim server. Therefore, the attacker can significantly magnify the effect of the DoS attacks on the victim server[13]. IoT devices are a perfect choice to perform DDoS attacks given the huge number of IoT devices that have access to the Internet and also their vulnerabilities to become compromised.

DoS and DDoS attacks could be performed by using the protocols of different layers in the Open Systems Interconnection (OSI) model. DDoS attacks that use the transport layer are the most common type of attacks where the attacker sends as many packets as possible to the victim server through UDP flooding, SYN flooding, etc. We also have other types of DDoS attacks that use the application layer in the OSI model. Attackers try to disturb the behavior of the victim server by tying up its every thread with slow requests. The slow-rate DDoS attacks could be performed by sending data to the victim server at a very slow rate, but fast enough to prevent the connection from getting timed out [14].

One of the most famous IoT-based DDoS attacks is caused by the Mirai botnet. This botnet could bring down the victim servers by infecting thousands of IoT devices and dramatically increase the network traffic directed toward the victim servers on the order of Terabit per second (Tbps), affecting millions of end-users [15, 16, 17]. Another example is the Reaper botnet which is a variant of the Mirai botnet and could infect 2.7 million IoT devices [18]. There are many other botnets that could perform DDoS attacks by using billions of IoT devices around the world and impact many victim servers, and end
users [19].

Given this dangerous and dramatically growing threat, we explore the use of machine learning (ML) techniques as the main tool to prevent such attacks [20, 21]. One of the most important gaps in the related papers that studies the DDoS detection mechanisms is the fact that all of them consider either the packet volume transmitted from IoT nodes or the packet flow timing while under attack to be significantly (orders of magnitudes) higher as compared to the IoT nodes' benign traffic. For instance, Meidan _et al._[22] presented an ML-based technique for detecting the DDoS attacks on a dataset containing the benign and attack traffic of nine IoT nodes. Figure (a)a shows the probability density of benign and attack traffic packet volume of an IoT node with ID XCS7_1003 in the Meidan _et al._[22] paper. As we can see, the packet volume transmitted from that IoT node while under attack is, on average, 1604 times higher than the packet volume of the IoT node when there is no attack happening. Furthermore, Sharafaldin _et al._[23] presented a DDoS dataset and also an ML-based technique for detecting DDoS attacks. Their dataset also includes a slow-rate DDoS attack type. Figure (b)b shows the probability density of benign and attack traffic mean flow inter-arrival time (IAT), i.e. the mean time between two packets sent in the flow. As we can see, the average of the mean IAT for the attack traffic is approximately 11 times higher than the benign traffic. These differences between the benign and attack traffic properties could give a huge benefit to the ML models for detecting DDoS attacks. However, in this study, we consider more futuristic DDoS attacks in which attackers can mimic the behavior of the IoT nodes' benign traffic. In fact, given the huge number of IoT devices that are currently available, the attackers could take control of millions of IoT nodes and, camouflage their attack by sending fewer packets from those IoT nodes with similar timing to the benign traffic, thus, disturbing the behavior of the victim server. This futuristic attack which is the focus of this study, is much more sophisticated than traditional attacks where the attacker could expose themselves by either sending many packets or slowing down the data transmission.

We have released an anonymized dataset containing real-trace data from an urban deployment of 4060 IoT devices that records their binary activity [24]. We further enhanced that dataset in our recent work [25] by adding the packet volume that each IoT device transmits at each timestamp where they are active. The basis for this emulation is grounded in our finding that a real urban IoT benign traffic can be well-modeled as a (truncated) Cauchy distribution (matching the observation of prior researchers that Ethernet traffic is well modeled by such a distribution [26]). We have also done a simple analysis of neural network models by only considering one architecture where each IoT node trains individual neural network models without using the correlation information from other nodes [25].

In this work, we further enhance the generation of our training dataset, in which we also include the correlation information of IoT nodes' activity in each recorded activity on the dataset. Our proposed "dataset+script" allows the injection of truncated Cauchy distributed attacks, and it also lets the user parametrize the difference between the benign and attack traffic volume. In this way, a diverse training scenario can be effectively generated to improve the training of neural network models. All attack traffic volumes are tuned by a parameter that we call "\(k\)" that determines both the location and scale of the truncated Cauchy distribution for attack traffic volume generation. We further extend our recent work by proposing four architectures to train neural network models for the IoT devices by considering all combinations of either using the correlation information of the IoT devices or not, and either (a) training one central model for all nodes or (b) training distributed individual models for each IoT device. By using the correlation information, each IoT node has access to the packet volume transmitted from other IoT nodes in addition to its packet volume. These architectures are named multiple models with correlation (MM-WC), multiple models without correlation (MM-NC), one model with correlation (OM-WC), and one model without correlation (OM-NC). Furthermore, we considered five different neural network models, namely multi-layered perceptron (MLP), convolutional neural networks (CNN), long short-term memory (LSTM), transformer (TRF), and autoencoders (AEN). We extensively analyzed the performance of proposed architectures by using the mentioned neural network models to decide what is the best architecture/neural network model for detecting DDoS attacks on IoT devices.

Our simulation results indicate that using the correlation information of the nodes significantly helps with detecting DDoS attacks, especially in the case that the attacker is camouflaging the attack by using the benign traffic packet volume distribution. Given the huge number of IoT devices that could potentially be used in performing DDoS attacks, using the correlation information of all nodes results in a massive number of features that neural network models need to learn from. Therefore, we investigated the methods to actively select the nodes for considering the correlation information of the nodes for training the neural network models. Our results indicate that using the Pearson correlation to actively select the nodes for training and prediction purposes, results in a good performance in terms of binary accuracy and F1 score for detecting the DDoS attacks as compared to using the correlation information of all nodes. We make the dataset, attack emulation, and training/testing neural network models and ar

Fig. 1: Benign/attack packet volume (figure left) and mean flow inter-arrival time (figure right) probability density of real DDoS attacks
chitectures script available as an open-source repository online at [https://github.com/ANRGUSC/correlation_aware_ddos_iot](https://github.com/ANRGUSC/correlation_aware_ddos_iot).

The rest of this paper is organized as follows: section II presents the related works that have been done in this area. In section III, we present the raw urban IoT dataset and its benign traffic characteristics; this section also introduces the modeling of the IoT benign traffic (defining the (truncated) Cauchy distribution) and builds the synthetic dataset defining the parameter that will regulate the relation/distance between benign and attack traffic. In section IV we illustrate the way we emulate attacks and also the procedure to create the general training dataset to be used for training, validating, and comparing different NN models and architectures to detect DDoS attacks deployed in the customized dataset. Section V presents the four architectures and five neural network models we used for this analysis. In section VI, we discuss new methods for considering the correlation information of IoT nodes while considering their constrained resources by actively selecting IoT nodes for sharing their information in correlation-aware architectures. Section VII presents the evaluation and analysis of the introduced models. Lastly, section VIII summarizes this work and proposes future research steps.

## II Related Works

Prior works have explored various machine learning models for detecting DDoS attacks. Most of the papers in this area considered the transport layer DDoS attacks, where attackers send as many packets as possible to the victim server to disturb its behavior. Doshi _et al._[27] generated the training data by setting up an environment with three IoT devices and simulated the Mirai Botnet attack. They have developed various classification models such as random forests, K-nearest neighbors, support vector machines, and neural networks to be run on network middleboxes such as routers, firewalls, or network switches to detect DDoS attacks, and concluded that almost all of the models could achieve 0.99 accuracy. Chen _et al._[28] used 9 IoT devices at a university campus to collect the data through IoT gateways and transmit the data to cloud servers via the software-defined network (SDN) switches. Then, they implemented a decision tree to run on the IoT gateways and SDN controllers in order to detect and block the DDoS attacks and achieved an F1 score of 0.99. Similarly, Mohammed _et al._[29] implemented a naive Bayes classification technique based on the NSL-KDD dataset on a central server and then tested their model using the traffic information captured from four SDN controllers and achieved an F1 score of 0.98. Syed _et al._[30] proposed an application layer DoS attack detection framework that uses a machine learning-based method on the Message Queuing Telemetry Transport (MQTT) brokers. They tested their framework by using the average one-dependence estimator (AODE), C4.5 decision trees, and multi-layer perceptron (MLP) machine learning models on three virtual machines and achieved 0.99 accuracy. Meidan _et al._[22] collected their training data by infecting nine commercial IoT devices and proposed N-BaIoT anomaly detection method that uses autoencoders to classify the benign and attacked IoT traffic with 0.99 accuracy.

Here we also mention some of the works that studied slow-rate DDoS detection mechanisms. Yungaicela-Naula _et al._[14] used CICDDoS 2017 [23], and CICDDoS 2019 [31] and utilized machine learning and deep learning methods such as support vector machine (SVM), K nearest neighbor (KNN), multi-layer perception (MLP), etc. to detect DDoS attacks. Their experiments showed that they could achieve up to 98% accuracy for the traditional DDoS attack and up to 95% accuracy for slow-rate DDoS attacks. Similarly, Nugraha and Murthy [32] proposed a hybrid CNN and LSTM neural network model that could achieve up to 99% accuracy on a dataset that they self-developed for slow-rate DDoS attacks. Muraleedharan and Janet [33] used a random forest (RF) classifier to detect slow-rate DDoS attacks, and their model could achieve up to %99 accuracy. Cheng _et al._[34] studied the performance of random forest, K nearest neighbor, naive Bayes, and support vector machine on a self-developed dataset for slow-rate DDoS attack detection on the network switches and controllers. Their analysis showed that ML models could achieve up to %99 accuracy. There are many other works that studied the DDoS detection on IoT devices using machine learning methods some of them are presented in table I.

In all the related works discussed above, we are observing that the datasets and features that they have used to do DDoS attack classification have very different properties for the attack and benign traffic, where attack traffic volume/timing is orders of magnitude higher than the benign traffic volume/timing. However, in this work, we use the tunable parameter \(k\) that can tune the packet volume being transmitted from IoT devices with regular timing as compared to the benign traffic and help the attacker to camouflage the attack in the benign traffic. Furthermore, most of the mentioned related works studied either a central neural network model to run on the cloud servers, SDN switches, etc., or a few of them studied the distributed neural network model to run on the IoT devices and did not provide a comparison for the distributed versus central architectures. However, in this work, we extensively analyzed both of these scenarios and compared their performance against each other. Furthermore, none of the papers mentioned above considered the correlation information of IoT nodes with relation to each other while designing the DDoS detection mechanisms. However, in this paper, we also study the architectures that are correlation-aware and consider the correlation information of the IoT nodes for detection purposes. Moreover, our paper proposes and uses a large-scale IoT DDoS dataset with more than 4000 nodes, and we introduce a training dataset that considers correlation information of the IoT nodes against other papers which use datasets that have a limited number of IoT nodes without having the correlation information of the nodes included. Table II presents an overview of datasets in this field with the respective number of IoT nodes used in creating the dataset.

## III Original and Benign Activity Datasets

The original (raw) data of the binary activity of the IoT nodes have been collected from the IoT nodes that were
deployed in an urban area. 1. The features provided in the raw dataset are the node ID, the latitude and longitude location of the IoT node, and the binary activity status of the IoT devices. A record of the IoT node appears on the raw dataset whenever there is an activity status change for an IoT node, i.e., after each change in the activity status of the IoT nodes, a new record will be added to the raw dataset. The raw dataset has 4060 nodes with one month's worth of data with no missing data points.

Footnote 1: The source of this data, originally presented in [24], has been anonymized for privacy and security reasons.

As mentioned before, we have activity status changes of the nodes in the raw dataset. This means that the nodes that have more changes in their activity will have a higher number of records in the raw dataset. Therefore, using the raw dataset for the purpose of training neural network models will provide a bias toward the behavior and information of the nodes that have the highest number of activity changes. In order to address this issue, we provided a script that takes the raw dataset with the record of activity status changes of the IoT nodes and also a time step, called \(t_{s}\), and generates a new dataset called benign dataset that presents the activity status of each IoT node every \(t_{s}\) seconds. Therefore, the activity status of each IoT node will be present in the benign dataset every \(t_{s}\) seconds, which helps the neural network models to learn the activity behavior of all IoT nodes at all time stamps instead of just learning the behavior of the IoT nodes that have the highest changes in their activity status. Furthermore, the python script can also get a beginning and ending date for generating the benign dataset in the case the IoT node has constrained resources to train neural network models on all the benign dataset.

As mentioned above, the benign dataset will only have the binary activity status of the IoT nodes. In order to enhance the dataset, we also add the packet volume that gets transmitted from the active IoT nodes at each time step. Meidan et al. [22] studied the DDoS attack detection on 9 IoT nodes in the real world and presented a dataset that contains the packet volume transmitted from each IoT node every 10 seconds both in the case that nodes are under attack and also the case that nodes are not under attack. We used the data related to a security camera IoT node with ID XCS7_1003 in the Meidan et al. [22] paper as a source for generating the benign/attack packet volume of the IoT nodes in our dataset. In the literature, various distributions have been used to estimate the packet volume of network traffic [52]. We analyzed 80 different distributions for estimating the packet volume of the IoT nodes based on the real information that we had for the security camera datatset that was provided in [22]. Among the 80 different distributions that we analyzed, the

\begin{table}
\begin{tabular}{l c c c c c} \hline
**Reference** & **Date** & **Number of Nodes** & **IoT specific/General** & **Binary activity or Traffic Volume** & **Benign/Attack traffic** \\ \hline DARPA 2000[45] & 2000 & 60 & general & traffic volume & both \\ CAIDA UCSD DDoS Attack 2007 [46] & 2007 & unclear & general & traffic volume & attack \\ Shiravi et. al. [47] & 2012 & 24 & general & traffic volume & both \\ CICDDoS 2017 [23] & 2017 & 25 & general & traffic volume & both \\ Median et. al. [22] & 2018 & 9 & IoT specific & traffic volume & both \\ CSE-CIC-IDS2018 on & AWS & 2018 & 450 & general & traffic volume & attack \\
[48] & & 2019 & 25 & general & traffic volume & attack \\ CLCDDS 2019 [31] & 2019 & unclear & IoT specific & traffic volume & both \\ The Bot-IoT Dataset (Univ. of U18W) [49] & 2020 & 42 & IoT specific & traffic volume & both \\ Ullah et. al [50] & 2020 & 400 & general & traffic volume & both \\ Erhan et. al. [51]. & 2020 & 4000 & IoT specific & binary activity & both \\ Hekmait et. al. [24] & 2021 & 4060 & IoT specific & traffic volume & both \\ \hline \end{tabular}
\end{table} TABLE II: Related Papers with IoT datasets

\begin{table}
\begin{tabular}{l l l l l} \hline
**Reference** & **Dataset** & **Detection Method** & **Centralized/Distributed** & **Inference Device** \\ \hline Doshi _et al._[27] & Self-developed & RF, KNN, SVM, MLP & Centralized & Network middle box \\ Chen _et al._[28] & Self-developed & DT & Centralized & SDN controller \\ Syed _et al._[30] & Self-developed & MLP, AODE, DT & Centralized & MQT broker \\ Meidan _et al._[22] & N-BaloT [22] & AEN & Distributed & IoT node \\ Liu _et al._[35] & Self-developed & RL & Centralized & SDN controller \\ Roopak _et al._[36] & Self-developed & MLP, CNN, LSTM & Centralized & Unclear \\ Guntlakshmi _et al._[37] & Self-developed & SVM & Centralized & Unclear \\ Mohammad _et al._[29] & NSL-KDD [38] & NB & Centralized & SDN controller \\ Zekri _et al._[39] & Self-developed & DT & Centralized & Cloud \\ Blaise _et al._[40] & CTU-13 [41] & MLP, SVM, RF, LR & Centralized & Unclear \\ Soe _et al._[42] & N-BaIoT [22] & MLP, DT, NB & Centralized & Unclear \\ Nugraha _et al._[43] & CTU-13 [41] & MLP, CNN, LSTM & Centralized & Unclear \\ Kumar _et al._[44] & Self-developed & RF, KNN, GNB & Centralized & IoT gateway \\ Yungaiecla _et al._[14] & CICDDoS [23],[31] & RF, SVM, KNN, MLP, CNN, GRU, LSTM & Centralized & Server \\ Cheng _et al._[34] & Self-developed & SVM, NP, KNN, RF & Centralized & Controller and Switch \\ \hline \end{tabular} 
\begin{tabular}{l} Where MLP: Multilayer Perceptron, CNN: Convolutional Neural Network, LSTM: Long Short Term Memory, \\ AEN: Autoencoder, RL: Reinforcement Learning, DT: Decision Tree, RF: Random Forest, KNN: K Nearest Neighbor, \\ SVM: Support Vector Machine, AODE: Average One-Dependence Estimator, NB: Naive Bayes, \\ GNB: Gaussian Naive Bayes, LR: Logistic Regression, GRU: Gated Recurrent Unit \\ \end{tabular}
\end{table} TABLE I: Selected papers with ML based methods for detecting DDoS attacks on IoT
Cauchy distribution fitted best to the packet volume activity of the security camera node in terms of having the minimum square error. Since the Cauchy distribution has an unusual nature due to being unbounded in value and not having a defined mean, we use a truncated Cauchy distribution instead with a low of 0, and a high of maximum packet volume observed in the real data of the security camera IoT node. Our analysis in finding truncated Cauchy distribution as the best distribution for estimating the packet volume of the IoT nodes is also compatible with the previous finding for modeling network traffic [26]. In order to generate the packet volume in the benign dataset, we will generate the packet volume transmitted from the IoT node whenever the node is active based on the fitted truncated Cauchy distribution. On the other hand, when the node is not active, the packet volume transmission from the IoT node will be zero. Note that in this paper, we are presenting an event-driven IoT dataset which means that the IoT nodes will transmit packets whenever they are active. Table III presents a few sample data points in the benign dataset.

In order to generate the packet volume of the IoT nodes when they go under attack, we define a new truncated Cauchy distribution with the following parameters:

\[x_{a} =(1+k)\cdot x_{b} \tag{1}\] \[\gamma_{a} =(1+k)\cdot\gamma_{b}\] (2) \[m_{a} =(1+k)\cdot m_{b} \tag{3}\]

where, \(x_{b}\), \(\gamma_{b}\), \(m_{b}\) refer to the location, scale, and maximum packet volume of the truncated Cauchy distribution of the benign traffic, respectively, while \(x_{a}\), \(\gamma_{a}\) and \(m_{a}\) are the location, scale, and maximum packet volume of the generated truncated Cauchy distribution of the attack traffic, respectively. \(k\) is the tunable parameter for generating packet volumes with higher location, scale, and maximum packet volume as compared to benign traffic where \(k>=0\). Note that when the DDoS attack gets performed by the attacker, the IoT nodes will become active and transmit packets, whether they were active or not previously. As we mentioned before, the related works in this area are only considering the cases where in the DDoS attack, either IoT nodes are sending as many packets as possible or slowing down the data transmission to the victim server in order to disturb its behavior. However, in this work, we are considering the case that the attacker can perfectly camouflage the attack with benign traffic. More specifically, for low \(k\) values, the attacker is trying to camouflage with background traffic since it is sending packets by using the benign traffic pattern of the IoT nodes, which is harder to detect the DDoS attack. On the other hand, when the attacker uses a higher value for \(k\), larger packet volumes will be transmitted to the victim server, but it will also be easier to get detected. While one could potentially use three different parameters for creating new \(x_{a}\), \(\gamma_{a}\), and \(m_{a}\), we use only one parameter \(k\) just for simplicity.

In order to better understand the behavior of the tunable parameter \(k\) and also compare the truncated Cauchy distribution estimation against the real packet volume distribution, here we present figures 1(a) and 1(b) that show the probability density function (PDF) and complementary cumulative distribution function (CCDF) for the real empirical benign traffic data in a blue color besides the truncated Cauchy distribution with \(k\) values of 0, 2, and 8. By assigning the \(k\) value to 0, the packet volume generated by the truncated Cauchy distribution should be similar to the real packet volume distribution. As we can see in figures 1(a) and 1(b), the truncated Cauchy distribution with \(k=0\) (red color) is well fitted to the real empirical packet volume distribution (color blue). By increasing the value of the tunable parameter, \(k\), we are basically increasing the location, scale, and maximum packet volume of the truncated Cauchy distribution, which results in having higher probabilities for larger packet volumes. Using high values for \(k\) results in transferring huge amounts of packets during the attack. In this scenario, the attacker can disturb the behavior of the victim server with fewer number of IoT nodes. However, using lower values of \(k\), although lower packet volumes will be generated by each IoT node during the attack, the attacker could use a large number of IoT nodes to transmit packets to the victim server and disturb its behavior.

## IV Attack Mechanism

This section presents how synthetic DDoS attacks are generated by using the IoT nodes, given the packet volume distribution presented in section III. Furthermore, the procedure for generating the general training dataset is also discussed in this section.

### _Generating DDoS attack_

In this paper, we synthetically generate a DDoS attack on the IoT nodes by setting all nodes which are under attack

\begin{table}
\begin{tabular}{|c|c|c|c|c|c|} \hline
**NODE** & **LAT** & **LNG** & **TIME** & **ACTIVE** & **PACKET** \\ \hline
1 & 33 & 40 & 
\begin{tabular}{c} 2021-01-01 \\ 23:00:00 \\ \end{tabular} & 0 & 0 \\ \hline
1 & 33 & 40 & 
\begin{tabular}{c} 2021-01-01 \\ 23:10:00 \\ \end{tabular} & 0 & 0 \\ \hline
1 & 33 & 40 & 
\begin{tabular}{c} 2021-01-01 \\ 23:20:00 \\ \end{tabular} & 1 & 9 \\ \hline
1 & 33 & 40 & 
\begin{tabular}{c} 2021-01-01 \\ 23:30:00 \\ \end{tabular} & 1 & 11 \\ \hline \end{tabular}
\end{table} TABLE III: Sample Data Points in Benign Dataset

Fig. 2: Real packet volume distribution vs truncated Cauchy distribution
to active status for the duration of the attack and assigning what packet volume to be transmitted at each time slot. Recall that since we have an event-driven IoT dataset, when the nodes go under attack, they will start transmitting packets, whether they have been active or not previously. In order to generate the packet volume transmitted while the nodes are under attack, we use the equations (1), (2), and (3) to define the truncated Cauchy distribution for a given tunable parameter \(k\) and sample in i.i.d fashion from that distribution. The DDoS attack performed by the attacker can have four different parameters to be set: the start time of the attack (\(a_{s}\)), the duration of the attack (\(a_{d}\)), the ratio of the nodes to be used by the attacker (\(a_{r}\)), and the tunable attack packet distribution parameter (\(k\)). In this way, we can synthesize various desired attacks that an attacker could potentially perform. Therefore, the neural network models will be able to learn the behavior of the nodes in different attack scenarios and predict what IoT nodes are under attack at what time.

### _Generating General Training Dataset_

Here we present the procedure for creating the _general_ training dataset that contains all features that may be used by different architectures proposed in section V-A. By performing the attack on the benign dataset, we will generate a labeled general training dataset for supervised machine learning training. At each sample of the general training dataset for a specific IoT node \(i\), we store the node ID, the time stamp, and also the packet volume that is being transmitted through that node. In addition to the packet volume of node \(i\) at each sample of the training dataset, we also store the packet volume for all other nodes in the general training dataset. This means that at each time stamp of the general training dataset, we will have access to the packet volume that is being transmitted through node \(i\) and all other nodes. Furthermore, in order to distinguish the records for each node, we also add one-hot encoding to the general training dataset. Finally, we will have the label of each sample that shows whether that sample represents the node being under attack or not. Note that having the record of the packet volumes that are being transmitted from all nodes at each sample of the general training dataset will help to understand the correlation behavior of the nodes and will be used in training the correlation-aware architectures. As mentioned before, we will propose four different architectures that either use the correlation information and one-hot encoding in training or not. This matter will be elaborated more in section V-A. Table IV represents a sample training dataset containing the information of only two nodes where N_1 and N_2 represent nodes 1 and 2, respectively, and P_1 and P_2 represent the packet volume of nodes 1 and 2, respectively. Note that, in order to detect the DDoS attack at each timestamp, we also need the information of the previous time slots to understand the behavior of the node through time. Therefore, the training dataset could be considered a time-series dataset. Thus, we stack the past \(n_{t}\) entries of each sample in the training dataset, to predict the attack status of that specific sample. In the python script provided, one could also tune the \(n_{t}\) based on the resources available for the IoT devices. Having higher values for \(n_{t}\) would help the neural network model to better understand the behavior of the IoT devices, but it also requires higher computation resources for training the neural network models.

## V Defense Mechanism

In this section, we present the architectures and neural network models that we used for detecting DDoS attacks. The performance comparison of these architectures and neural network models are presented in section VII.

### _Architectures_

We introduce four different architectures to train the neural network models based on whether to use the correlation information of nodes' activity or not and whether to train distributed individual models for each IoT node or train one central model for all nodes. Here we present the details of each architecture:

* **Multiple models without correlation (MM-NC)**: In this architecture, we train individual neural network models for each IoT node. Therefore, each node will have one model trained just for itself. Furthermore, we do not use the correlation information of nodes' activity, i.e., we only use the packet volume of each node through time in order to train the models and do not include the packet volume of other nodes in the training dataset. Since we are training individual models for each node, we will also not use one-hot encoding features from the general training dataset in this architecture.
* **Multiple models with correlation (MM-WC)**: In this architecture, we train individual neural network models for each IoT node. Therefore, each node will have one model trained just for itself. Furthermore, we use the correlation information of nodes' activity, i.e., in addition to each node's packet volume, we also use the packet volume of other nodes to capture the correlation information of nodes' activity in training. Since we are training individual models for each node, we will also not use one-hot encoding features from the general training dataset in this architecture.
* **One model without correlation (OM-NC)**: In this architecture, we train one neural network model for all IoT nodes. Therefore, there will be a central node/server that trains the neural network model, and all IoT nodes will use that model for detecting DDoS attacks. Furthermore, we do not use the correlation information of nodes' activity, i.e., we only use the packet volume of each

\begin{table}
\begin{tabular}{|c|c|c|c|c|c|c|} \hline
**Node** & **Time** & **N\_1** & **N\_2** & **P\_1** & **P\_2** & **Attacked** \\ \hline
1 & 2021-01-01 & 1 & 0 & 12 & 9 & 0 \\ \hline
1 & 2021-01-01 & & & & & & \\  & 23:30:00 & 1 & 0 & 13 & 8 & 1 \\ \hline
2 & 2021-01-01 & 0 & 1 & 9 & 12 & 0 \\ \hline
2 & 2021-01-01 & & & & & & \\  & 23:30:00 & 0 & 1 & 8 & 13 & 1 \\ \hline \end{tabular}
\end{table} TABLE IV: Sample Data Points in General Training Dataset
node through time in order to train the models. Since we are training one central model for all nodes, in order to distinguish the information of each node from other nodes, we use one-hot encoding from the general training dataset.
* **One model with correlation (OM-WC)**: In this architecture, we train one neural network model for all IoT nodes. Therefore, there will be a central node/server that trains the neural network model, and all IoT nodes will use that model for detecting DDoS attacks. Furthermore, we use the correlation information of nodes' activity, i.e., in addition to each node's packet volume, we also use the packet volume of other nodes to capture the correlation information of nodes' activity in training. Since we are training one model for all nodes, in order to distinguish the information of each node from other nodes, we use one-hot encoding from the general training dataset.

Using the correlation information of the nodes in the MM-WC and OM-WC architectures could potentially help the neural network models to better predict the DDoS attack since attackers usually use many IoT devices for performing DDoS attacks. Using one model for training the neural network models in the OM-WC and OM-NC architecture will potentially learn the general behavior of the nodes' activity and inactivity better while the MM-WC and MM-NC architectures that use individual models for training neural network models for each node could better personalize the model for each IoT node based on their behavior.

### _Neural Network Models_

For each architecture discussed in section V-A, we use five different neural network models to do binary classification for detecting the DDoS attack on the IoT nodes:

* **Multilayer Perceptron (MLP)**: This is the simplest model of the feed-forward artificial neural network that consists of one input layer, one output layer, and one or more hidden layers [53]. In this paper, the input layer is followed by one dense layer with 5 neurons and Rectified Linear Unit (ReLU) activation. In the MM-WC and OM-WC architectures, the dense layer also has a 30% dropout rate, while in the MM-NC and OM-NC architectures, the dense layer does not perform dropout. The output is a single neuron with the Sigmoid activation function.
* **Convolutional Neural Network (CNN)**: CNNs are similar to MLP models but use mathematical convolution operation in at least one of their hidden layers instead of general matrix multiplication [54]. In this paper, the model's input layer is followed by one 1-dimensional convolution layer with 5 filters, kernel size of 3, and ReLU activation function. In the MM-WC and OM-WC architectures, the CNN layer also has a 30% dropout rate, while in the MM-NC and OM-NC architectures, the CNN layer does not have a dropout rate. The CNN layer is followed by a 1-dimensional max-pooling layer with a pool size of 2, and then a flattened layer. Finally, the output is a single neuron with the Sigmoid activation function.
* **Long Short-Term Memory (LSTM)**: This is a neural network model that can process a sequence of data and is a suitable choice for time-series datasets [55]. In this paper, the model's input layer is followed by one LSTM layer with 4 units and a hyperbolic tangent activation function. In the MM-WC and OM-WC architectures, the LSTM layer also has an L2 regularization factor of 0.3, while in the MM-NC and OM-NC architectures, the LSTM layer does not have an L2 regularization. Finally, the output is a single neuron with the Sigmoid activation function.
* **Transformer (TRF)**: Transformer models are a neural network model that solely relies on the attention mechanism to calculate dependencies between the input and output [56]. This model has an encoder and decoder, but for the purpose of binary classification, we will only need the encoder part. In this model, the input layer is followed by a multi-head attention layer with one attention head, and the size of each attention head for query and key is 1. In the MM-WC and OM-WC architectures, the multi-head attention layer also has an L2 regularization factor of 0.3, while the MM-NC, and OM-NC architectures, do not have L2 regularization. The multi-head attention layer is followed by the global average pooling layer. Finally, the output is a single neuron with the Sigmoid activation function.
* **Autoencoder (AEN)**: Autoencoders are a type of neural network model that consists of two parts. The first part tries to encode the input into a compressed representation and the second part tries to decode the compressed representation into the original input [57]. In fact, this model tries to learn a meaningful compressed representation of the input. In this paper, the input layer is followed by an encoder that consists of five dense layers with 256, 128, 64, 32, and 16 neurons with a ReLU activation function, each followed by batch normalization. The decoder consists of five dense layers with 16, 32, 64, 128, and 256 neurons with a ReLU activation function, each followed by batch normalization. Finally, the output dimension will be the same as the input layer. For the binary classification, the encoder is followed by a dense layer with 8 neurons and a ReLU activation function, and the output will be a single neuron with a Sigmoid activation function. By using the AEN model, we essentially train the encoder to encode our input dataset into a latent space. Then, we design a classification model that gets this latent space as input and predicts whether the latent features show malicious behavior or not.

Figure 3 also presents the architectures discussed above. Given that the correlation-aware architectures have access to the correlation information of the IoT nodes, i.e. MM-WC and OM-WC, we used regularization techniques such as dropout and L2 regularization methods to address the overfitting issues that we observed while training the neural network models. This is worth mentioning that the parameters and hyperparameters for designing/training the neural network model has been selected through a grid search that provides the
highest performance. Furthermore, since we are dealing with an unbalanced dataset to do binary classification, we should weigh more the minority class in training the neural network models and also intelligently set the initial bias of the layers. We used the practical method presented in [58] to set the class weights and also initial bias. The following formulation is used to define the weights for each class and also the initial bias of the layers:

\[w_{n} =\frac{pos+neg}{2\ neg} \tag{4}\] \[w_{p} =\frac{pos+neg}{2\ pos}\] (5) \[b_{0} =\log(\frac{pos}{neg}) \tag{6}\]

where, \(w_{n}\) and \(w_{p}\) represents the weight for the negative(not attacked) and positive(attacked) class, respectively. \(b_{0}\) represents the initial bias of the layers. \(neg\) and \(pos\) represent the total number of negative and positive samples, respectively.

## VI IoT Nodes with Constrained Resources

The architectures introduced in section V-A that use correlation information of the nodes, i.e. MM-WC and OM-WC, need to combine the information of _all_ nodes in order to train the neural network models. In fact, the packet volume of _all_ of the nodes will be used as an input to the neural network models. Using the correlation-aware architectures, results in a massive number of input features to be fed to the neural network models and make it infeasible to use in real-world scenarios with a huge number of IoT devices that have constrained resources and computation power. In order to address this issue, we propose that each IoT node uses a fraction of the other IoT nodes' information as input to their neural network models. As we can see in figure 4, we could either use all of the nodes' traffic information (figure 3(a)) or use selected nodes' information (figure 3(b)) to be shared with a specific node for the purpose of training neural networks and predicting DDoS attacks. By actively selecting nodes, we can hugely save time and computation in training neural networks and predicting DDoS attacks on IoT nodes with constrained resources.

In this section, we provide the following five heuristic solutions to select important IoT nodes for sharing their information with a specific IoT node in order to train the neural network models in the case of using architectures that utilize the correlation information:

* **All nodes**: In this method, we use the packet volume of all nodes for training the neural network models. This is the default method that has been discussed in previous sections, and we expect to have the best performance due to using the information of all nodes at the cost of needing high computation power.
* **Pearson correlation**: In this method, we use Pearson correlation to calculate the correlation of the behavior of the nodes' activity based on the packet volume that they are transmitting throughout the day. More precisely, for each node \(i\), we calculate the Pearson correlation of node \(i\) with all other nodes and find the top \(n\) nodes that have the highest correlation with the node \(i\). Finally, in training the correlation-aware architectures for each node, we only use the information of top \(n\) nodes that have the highest Pearson correlation with node \(i\), in addition to the information of node \(i\).
* **SHAP**: In this method, we use SHapley Additive exPlanations (SHAP) [59] to determine which features are the most important in making a decision of whether a node is under attack or not. More precisely, after training a neural network model for node \(i\) by using the information of all nodes, we run SHAP method to determine what are the top \(n\) features in making the decision for node \(i\). Then, we will train a new neural network model that only uses the information of the top \(n\) features in making the decision. Note that, in this method, we always need to train the neural network model on all of the nodes first and then determine what nodes provide the most information for the detection by using SHAP. Then, we need to retrain the neural network models by using the top \(n\) features. This is not a practical solution to be used in the real

Fig. 4: Using all nodes vs selected nodes for sharing information in correlation-aware architectures

Fig. 3: Neural Network Models
world, but we also implemented this method to compare it against other solutions.
* **Nearest Neighbor**: In this method, we use the euclidean distance of the nodes to calculate the correlation of the behavior of the nodes' activity based on their distance. More precisely, for each node \(i\), we calculate the euclidean distance of node \(i\) with all other nodes and find the top \(n\) nodes that are closest to the node \(i\). Finally, in training the correlation-aware architectures for each node, we only use the information of the top closest \(n\) nodes in addition to the information of node \(i\).
* **Random**: In this method, in order to train the correlation-aware architectures for node \(i\), we randomly select \(n\) other nodes and use their information in addition to the node's \(i\) information in order to train the neural network models. Then, we will repeat this process for the desired number of times and calculate the average of the evaluation metrics of all runs. The random method gives us a baseline to compare it against the other solutions mentioned above.

## VII Neural Network Models and Architectures Evaluation

In this section, we evaluate the proposed neural network models and also architectures by doing extensive simulations to analyze each architecture/model's performance in different scenarios.

### _Experiment Setup_

In our simulations, we used a time step of 10 minutes, i.e. \(t_{s}=600seconds\), for creating the benign dataset by using the method mentioned in section III, but one could use other desired time steps by using the script provided. The introduced enhanced dataset has 4060 IoT nodes. In this subsection, we randomly selected 50 nodes to do the simulations. Here we present the activity analysis of the full dataset with 4060 nodes against the 50 random nodes that we have chosen from the dataset for the purpose of evaluating neural network models.

Figure 5 presents the mean number of active nodes in the benign dataset versus the time of the day by considering all nodes and also the 50 randomly selected nodes. As we can see, up to 70% of the nodes, get activated around the middle of the day, but by midnight only about 20% of the nodes are active. We can also see that the 50 randomly selected IoT nodes have similar activity behavior as compared to the activity behavior of all nodes.

Figure 6 shows the probability density function of nodes' mean activity/inactivity throughout the day (from 8 AM to 8 PM) and night (from 8 PM to 8 AM) for all nodes and also the 50 randomly selected nodes. We observe that nodes are primarily active for around 100 minutes and also mostly inactive for around 80 minutes during the day. During the night, nodes are primarily active for around 80 minutes, but they are mostly inactive for around 220 minutes. Another takeaway from figure 6 is that the activity and inactivity behavior of the 50 randomly selected nodes in the day and night is also similar to the behavior of all nodes. Therefore based on our observation from figures 5 and 6, the 50 randomly selected IoT nodes closely represent the behavior of all nodes in the dataset, and we can safely extend our simulation findings by using the results that we get by training the neural network models and architectures over the 50 randomly selected nodes.

The whole dataset has one month's worth of data, and in this simulation, we used 4, 1, and 3 different days of the dataset for the training, validation, and testing of the neural network models, respectively. As we discussed in section IV-A, we can synthesize the DDoS attacks by providing the attack properties to the python script. The properties of the DDoS attacks that we used in this simulation are the following:

* The attacks can start at 2 AM, 6 AM, and 12 PM. We chose different start times for the attacks to make sure the models get trained to predict the DDoS attacks even

Fig. 5: Active Nodes Percentage vs Time

Fig. 6: Probability density function (PDF) of nodes mean activity/inactivity for day and night
at the high trafficked times of the day like at noon and do not have a bias towards the time of the day that the DDoS attacks may start.
* The duration of the attacks could be 4 hours, 8 hours, or 16 hours. We used different hours of attacks to make sure the model could predict both short and long duration DDoS attacks.
* The ratio of the nodes that go under attack could be selected from 0.5 and 1. We chose different attack ratios due to the fact that sometimes the attackers do not use all the compromised IoT devices to perform the DDoS attack.
* Attack parameter, \(k\), which tunes the packet volume that is being transmitted from the IoT nodes that go under attack, could be chosen from 0, 0.1, 0.3, 0.5, 0.7, and 1. When \(k\) is close to 0, the attacker is using the benign traffic distribution to transmit packets to the victim server, which makes it hard to get detected. On the other hand, when \(k\) is close to 1, the packets transmitted from the attacked nodes will be higher, and therefore more harm is caused to the victim server but potentially easier to get detected.

Note that, in order to generate the attack and general training dataset, we use the combination of all possible attack scenarios by using the values of the attack properties discussed above in order to consider all possible combinations for training the neural network models.

In order to generate the training dataset, we considered a time window of 10, i.e. \(n_{t}=10\), which means the neural network models will make predictions based on the information on the past 10 time slots of each sample. After generating the training datasets, it has been shuffled to not have prediction bias towards the time of the day. All proposed neural network models have been trained for 3 epochs using 32 batches.

### _Binary Classification Metrics_

In this subsection, we present the performance of the different architectures and neural network models in terms of their binary accuracy, F1 score, and area under curve (AUC) versus the tunable parameter, \(k\), over the testing dataset. Furthermore, we also present the receiver operating characteristic (ROC) curve for \(k=0\) to compare the performance of the different architectures and different neural network models for the hardest scenario of detecting DDoS attacks while attackers use benign traffic distribution to generate packets. Note that, in order to calculate the mentioned performance metrics for each value of \(k\), we are calculating the average of the model's prediction performance over all other attack properties, such as attack start time, duration, and the ratio of the nodes that go under attack. By these figures, we can effectively compare the performance of different architectures and neural network models over each value of \(k\) and conclude what architecture/model performs the best for each scenario.

Figure 7 presents the performance of the multiple models with correlation architecture, i.e. MM-WC, across different neural network models discussed in section V-B. As we can see, LSTM neural network model has superior performance as compared to other models almost in all metrics, with an F1 score of between 0.82 and 0.87. TRF has a slightly lower performance as compared to the LSTM model. Following them, CNN and MLP are performing best, with an F1 score between 0.8 and 0.83. AEN has the worst performance for the MM-WC architecture, with an F1 score between 0.39 and 0.45. This shows us that AEN can not learn the behavior of the benign traffic due to the many input features of the MM-WC architecture. This matter is elaborated more at the end of this subsection.

Figure 8 presents the performance of the multiple models without correlation architecture, i.e. MM-NC, across different neural network models. As we can see, the performance of all neural network models is increasing with higher values of \(k\). LSTM neural network model has superior performance as compared to other models with an F1 score between 0.35 and 0.85. Following the LSTM model, CNN, MLP, and TRF are doing best with similar performance. Again, AEN has the worst performance for the MM-NC architecture as compared to other models with an F1 score between 0.51 and 0.65. One thing to note is that AEN has a better performance by using MM-NC architecture as compared to the MM-WC architecture. Furthermore, we are observing that AEN is showing superior performance for the case \(k=0\) as compared to other models for the F1 score metric but doing similar or worse for other metrics, i.e. binary accuracy, AUC, and ROC curve. The improved performance of the AEN model in the MM-NC architecture as compared to the MM-WC architecture shows that fewer input features are more suitable for the AEN architecture that can better learn the behavior of the benign traffic and compare it to the attack traffic for the DDoS attack detection.

Figure 9 presents the performance of the one model with

Fig. 7: Compare different neural network models’ performance by using the multiple models with correlation (MM-WC) architecture
correlation architecture, i.e. OM-WC, across different neural network models. As we can see, TRF neural network model has superior performance as compared to other models, with an F1 score between 0.82 and 0.85. Following that, the LSTM model is performing best, with an F1 score between 0.79 and 0.81. Finally, MLP and CNN are showing similar performance with an F1 score between 0.76 and 0.78. Again, AEN has the worst performance for the OM-WC architecture. This figure, similar to the MM-WC architecture, supports the fact that AEN could not handle many input features to the model. We can also observe that the TRF model can handle more complex architectures, such as OM-WC architecture, where we have many input features to the neural network model as compared to other models.

Figure 10 presents the performance of the one model without correlation architecture, i.e. OM-NC, across different neural network models. As we can see, the performance of all neural network models is increasing with higher values of \(k\). We can again observe that LSTM neural network model has superior performance as compared to other models, with an F1 score between 0.4 and 0.86. Following that, CNN, TRF, and MLP show similar performance with an F1 score between 0.39 and 0.85. Again, AEN has the worst performance for the OM-NC architecture.
although we see worse performance for the models as compared to the architectures that use correlation information, the metrics of the models improve when we go from low values of \(k\) to higher values of \(k\). This is due to the fact that having a higher value for \(k\) means the attackers are using higher packet volume for generating the DDoS attacks, which makes it easier for the neural network models to detect the attacks even without using the correlation information of other nodes' activity.
* The AEN model is performing worst as compared to all other models, almost in all scenarios. This is mainly attributed to the fact that for training the autoencoder model, we use the benign dataset traffic to learn the latent space attributed to the behavior of the nodes which are not under attack and then compare that latent space against the situations that nodes are under attack. Since we are training only on four days' worth of data, the autoencoder can not fully learn a meaningful latent space of the benign behavior of the nodes. The worst performance for the AEN model showed to be in the MM-WC architecture. In MM-WC architecture, we have very few samples since we are only using the samples of each IoT node individually as compared to OM-WC and OM-NC architectures, where we are using the samples of all nodes. Furthermore, MM-WC architecture also has considerably large input features due to using correlation information. Therefore, the AEN model has a hard time learning the behavior of the benign traffic and distinguishing it from the attack traffic. In this simulation, we used only 4 days' worth of data for training the models, but we expect that with increasing the number of training days, the AEN model should also perform better by taking in more samples of the benign traffic.

Since LSTM and TRF are performing better as compared to other models, here we also present the performance of only using either LSTM or TRF model against different architectures.

Figure 11 presents the performance of the LSTM model across different architectures. As we can see, the architectures that do not use correlation information, i.e. MM-NC and OM-NC, show poor performance with an F1 score around 0.35 for low values of \(k\). Their performance increase with higher values of \(k\) with an F1 score around 0.85 which is similar to the performance of architectures that use correlation information, i.e. MM-WC and OM-WC. Furthermore, we can also observe that MM-WC architecture is performing the best in almost all values of \(k\) as compared to other architectures with an F1 score around 0.82 which is similar to the performance of architectures that use correlation information, i.e. MM-NC and OM-NC, show poor performance with an F1 score around 0.4 for low values of \(k\). Their performance increase with higher values of \(k\) with an F1 score around 0.82 which is similar to the performance of architectures that use correlation information, i.e. MM-WC and OM-WC. Furthermore, we can also observe that both MM-WC and OM-WC architectures are showing similar good performance for all values of \(k\) as compared to other architectures.

All in all, we observe that using LSTM/MM-WC and TRF/OM-WC architecture provides the best and also similar performance. Depending on the application, one could use either of these models. In this paper, we prefer the LSTM/MM-WC architecture due to having distributed/individual models for each IoT node as compared to TRF/OM-WC architecture, where we only have one central model. Having a central model is not tolerant to failure, and the IoT system could hurt if that central model fails to do the DDoS detection. On the other hand, by having a distributed/individual model for each IoT node, the IoT system can survive even if some of the nodes

Fig. 11: Compare different architectures’ performance by using the LSTM neural network model

Fig. 12: Compare different architectures’ performance by using the TRF neural network model
fail to do the DDoS detection. In the rest of the paper, we will only consider the LSTM/MM-WC for further analysis.

### _DDoS Attack Prediction Analysis_

In this subsection, we present the DDoS attack prediction analysis by using the LSTM model and MM-WC architecture to see how fast and correctly in the real world our trained model could correctly predict the attacks.

Figures 12(a) and 12(b) show the attack prediction for the case \(k=0\) and \(k=1\), respectively. The scenario that we selected for this figure is that the attacks start at 12 PM, which is the busiest time of the day when most of the nodes are active. The attack duration lasts for 16 hours. Each figure shows the ratio of the nodes that are under attack versus the time of the day. The ground truth for attacks (Labeled as True with the color blue) shows what ratio of the nodes are truly under attack. The model's prediction of true positive (labeled as TP with the color orange) shows the ratio of the nodes that are correctly predicted by the model to be under attack. The model's prediction false positive (labeled as FP with the color green) shows the ratio of the nodes that are mistakenly predicted by the model as being under attack. As we can see, for both \(k=0\) and \(k=1\), we are seeing the model is correctly predicting the nodes that are under attack with high accuracy. For the case of \(k=1\), we are seeing a slightly better prediction of the attack while it is happening. This is due to the fact that for higher values of \(k\), the attacker is sending higher packet volume as compared to lower values of \(k\), and it would be easier for the model to detect the attack.

Based on our analysis, the LSTM model using the MM-WC architecture could detect 100% of the attack sessions described in section IV-A. In fact, it could predict at least one of the IoT nodes which are truly under attack to have abnormal behavior. Furthermore, on average, 88% of the IoT nodes that are truly under attack in all attack scenarios could correctly be predicted to have abnormal behavior. Additionally, on average, 92% of the nodes which are truly not under attack could be truly predicted to have benign behavior. In terms of falsely predicting a node to be under attack, on average, the false positive detection lasts around 5.39 time slots. Furthermore, in terms of missing nodes that are truly under attack, on average, the false negative detection lasts around 4.41 time slots.

To further analyze the performance of the proposed detection mechanism, we created a new testing dataset that varies the ratio of nodes that goes under attack from 0.1 to 1. Figure 14 shows the F1 score of the LSTM model using MM-WC architecture versus the ratio of the nodes that are under attack, i.e. \(a_{r}\). We have separated the different attack scenarios by considering the four cases of \(k=0/1\) and \(a_{d}=4/16hours\). As we can see, the scenario of \(k=1\) and \(a_{d}=16hours\) have the highest F1 score since the attacker is transferring many packets from the IoT nodes, and it is easier to get detected. On the other hand, when \(k=0\), for both \(a_{d}=4/16hours\), we can see that the performance of the detection model is suffered due to the attacker camouflaging the attack traffic with benign traffic, especially when also the attacker is using a very low number of IoT devices, i.e. \(a_{r}=0.1\) to perform the attack. Note that, the case of using a low number of IoT devices to perform the attack, i.e. \(a_{r}=0.1\), and also a small attack packet volume \(k=0\) is not very realistic due to the fact that although the attacker is sending some extra packets to the victim server, it can not essentially disturb the behavior of the victim server. Note that we believe having low performance in detecting the DDoS attacks in such mentioned not realistic scenarios is not a good way of analyzing the performance of our proposed model. However, we provided this simulation result just for the comprehensiveness of this study.

### _IoT Nodes with Constrained Resources_

Based on the simulation results provided in section VII-B, using the correlation information of the IoT nodes provides insightful information for the neural network models to make a much better prediction, especially for the case the attacker camouflages the attack in the benign traffic, i.e. \(k=0\). As we mentioned in section VI, using the correlation information of _all nodes_ is infeasible in the real world, given the huge number of IoT devices that could go under attack. Therefore, we introduced five methods in section VI to compare different techniques for incorporating the correlation information of the IoT nodes that have the highest impact on correctly

Fig. 14: F1 Score vs ratio of the nodes under attack (\(a_{r}\))

Fig. 13: Attack prediction vs time on the testing dataset for the case that attack starts at 12 pm for 8 hours over all IoT nodes.
predicting the DDoS attacks. In this subsection, we present the performance of different methods by using the information of 5 nodes, including the node that is running the model's prediction. Furthermore, we only consider the LSTM neural network model and MM-WC architecture as our detection method.

One of the methods mentioned in section VI uses the most important features in the training dataset based on the SHAP values to figure out the top 5 most important nodes. Figure 15 shows the top 5 important features in training the LSTM/MM-WC model for node 1159. The left vertical axis shows the feature names, and the features are sorted based on their importance in descending order. The right vertical axis shows the color heat map associated with the actual value of the feature, i.e. color shows whether the actual value of the feature is high (red color) or low (blue color) for each sample in the testing dataset. The horizontal axis shows the impact of each sample on the neural network model's prediction. In fact, higher(lower) values in the horizontal axis show that a sample has a higher(lower) prediction value. In our scenario, a higher prediction value shows an attack is happening, and a lower prediction value shows an attack is not happening. Figure 15 supports the fact that having a high packet volume (feature value) for node 1159, has a high impact (SHAP value) on predicting a DDoS attack happening by using node 1159. Furthermore, using the packet volume of the other four nodes, namely 3167, 33989, 23093, and 19159 helped the neural network model trained by node 1159 to predict whether a DDoS attack was happening or not. Therefore, in the SHAP method discussed in section VI, in order to train the LSTM neural network based on the MM-WC architecture for node 1159, we will only use the correlation information from nodes 1159, 3167, 33989, 23093, and 19159. We repeat the same procedure for other nodes to determine the top 5 important features for training the LSTM/MM-WC model for each node.

Figure 16 compares the performance of the different techniques discussed in section VI by using the LSTM/MM-WC model and also considering the top 5 important nodes. Figure 16 supports the assumption that using the information of all nodes provides higher performance with an F1 score of between 0.82 and 0.87 as compared to other techniques, which only use the information of 5 nodes. Furthermore, we can see that using the Pearson correlation provides higher performance as compared to the SHAP, random, and nearest neighbor method with an F1 score degradation of up to 5% as compared to using the information of all nodes. Using the SHAP technique to determine the most important features has a slightly lower performance as compared to the Pearson correlation method. Interestingly, the nearest neighbor method shows the worst performance of all. With the help of these results, we can conclude that by using the Pearson correlation, we can detect DDoS attacks over a large number of IoT devices by using the correlation information of only a fraction of IoT nodes with a slight degradation in performance.

### _DDoS Detection Performance Analysis over All nodes In The Dataset_

In this subsection, we analyze the performance of the proposed DDoS detection mechanism over _all_ nodes of the dataset. For this purpose, we assume all nodes are randomly distributed into groups of 50 nodes, i.e., each group of 50 nodes can have access to the information of other nodes in that group. In total, we will have 81 groups of 50 IoT nodes. We use the LSTM neural network model and the MM-WC architecture. We also consider the _All Nodes_ and _Pearson Correlation_ methods for training and testing the neural network models that were introduced in section VI. Figure 17 shows the average performance of the LSTM/MM-WC model by using both _All Nodes_ and _Pearson Correlation_ method over different attack packet distribution parameter, i.e. \(k\). As we can see, if we use the correlation information of all 50 nodes in each group, we will have an F1 score between 0.81 and 0.85 while using the _Pearson Correlation_ method, we are observing up to 5% lower performance in F1 score as compared to the _All Nodes_ method.

## VIII Conclusion

In this paper, we presented a machine-learning-based mechanism for detecting DDoS attacks in the ever-growing IoT

Fig. 16: Compare the performance of different methods for selecting subset of the nodes for training the LSTM neural network model

Fig. 15: Feature importance analysis based on the LSTM/MM-WC model for node 1159
systems. We used our previously published dataset that is based on real urban IoT activity data and emulated benign and attack traffic by using truncated Cauchy distribution. We introduced a new general training dataset that incorporated the correlation information of IoT nodes to be used by correlation-aware architectures for model training. Four new architectures, namely MM-WC, MM-NC, OM-WC, and OM-NC introduced that consider the case of either using a central or distributed neural network model and also consider either using the information of individual IoT nodes or using the correlation information of the IoT nodes. Furthermore, we studied the performance of five neural network models, namely MLP, CNN, LSTM, TRF, and AEN, by using the combination of all proposed architectures. Our extensive simulation results indicated that LSTM/MM-WC and TRF/OM-WC models have the best performance as compared to other combinations of architectures and neural network models. Furthermore, we observed that the architectures that use the correlation information of the nodes, i.e. MM-WC and OM-WC, have better performance as compared to the architectures that do not utilize the correlation information, especially in the case the attacker tries to camouflage the attack by mimicking the behavior of the benign traffic. Since the attacker could potentially use a massive number of IoT devices to perform a DDoS attack, it is not feasible to use the correlation information of all nodes for training neural network models. Therefore, we introduced heuristic solutions that actively select fewer number of IoT nodes for the purpose of training correlation-aware neural network models and detecting DDoS attacks. Our analysis indicated that selecting IoT nodes based on having the highest Pearson correlation for the purpose of training the correlation-aware architectures has the best performance as compared to other solutions. In future work, we plan to consider more complex detection architectures by using a collaborative method that uses IoT nodes, edge devices, and cloud servers. Furthermore, we want to explore the case of dynamically training and updating neural network models to make them capable of detecting DDoS attacks that use different behavior through time.

## IX Acknowledgments

This material is based upon work supported by Defense Advanced Research Projects Agency (DARPA) under Contract No. HR001120C0160 for the Open, Programmable, Secure 5G (OPS-5G) program. Any views, opinions, and/or findings expressed are those of the author(s) and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government.
