# Evolution of MAC Protocols in the Machine Learning Decade: A Comprehensive
Survey

Mostafa Hussien, Islam A.T.F. Taj-Eddin, Mohammed F. A. Ahmed,
Ali Ranjha, Kim Khoa Nguyen  Mohamed Cheriet Manuscript is under processing
and not final.

###### Abstract

The last decade, (2012 - 2022), saw an unprecedented advance in machine
learning (ML) techniques, particularly deep learning (DL). As a result of the
proven capabilities of DL, a large amount of work has been presented and
studied in almost every field. Since 2012, when the convolution neural
networks have been reintroduced in the context of ImagNet competition, DL
continued to achieve superior performance in many challenging tasks and
problems. Wireless communications, in general, and medium access control (MAC)
techniques, in particular, were among the fields that were heavily affected by
this improvement. MAC protocols play a critical role in defining the
performance of wireless communication systems. At the same time, the community
lacks a comprehensive survey that collects, analyses, and categorizes the
recent work in ML-inspired MAC techniques. In this work, we fill this gap by
surveying a long line of work in this era. We solidify the impact of machine
learning on wireless MAC protocols. We provide a comprehensive background to
the widely adopted MAC techniques, their design issues, and their taxonomy, in
connection with the famous application domains. Furthermore, we provide an
overview of the ML techniques that have been considered in this context.
Finally, we augment our work by proposing some promising future research
directions and open research questions that are worth further investigation.

###### Index Terms:

Machine learning, deep learning, reinforcement learning, medium access control
protocols, wireless communications, Internet of things.

##  I Introduction

Accronym | Meaning
---|---
ANN | Artificial Neural Networks
BSN | Body Sensor Network
CR | Cognitive Radio
CAP | Contention Access Period
COP | Contention Only Period
CSMA/CA | Carrier Sensing Multiple Access/Collision Avoidance
CFS | Correlation Feature Selection
DRL | Deep Reinforcement Learning
DL | Deep Learning
DCF | Distributed Coordination Function
DSA | Dynamic Spectrum Access
HDP-HMM | Hierarchical Dirichlet Process Hidden Markov Model
HS | Harmony Search
IPI | Inter-Packet Interval
IDS | Intrusion Detection System
KNN | K Nearest Neighbour
LA-MAC | Load Adaptive Medium Access Control
LSPI | Least Square Policy Iteration
MANET | Mobile Ad-hoc NETworks
MAC | Medium Access Control
M2M | Machine to Machine
MDP | Markov Decision Process
ML | Machine Learning
MAB | Multi-Armed Bandit
NN | Neural Networks
NLP | Natural Language Processing
PDR | Packet Delivery Ratio
PU | Primary User
PDS | Post Decision State
QoS | Quality of Service
QoE | Quality of Experience
QNN | Q-Neural Network
RTT | Round Trip Time
RL | Reinforcement Learning
RSS | Received Signal Strength
RF | Random Forest
SVM | Support Vector Machines
SMO | Sequential Minimal Optimization
SAML | Self Adapting MAC Layer
SU | Secondary User
TOP | Transmission Only Period
TSCH | Time-Slotted Channel Hopping
TD | Temporal Difference
TS | Thompson Sampling
TDMA | Time Division Multiple Access
WMN | Wireless Mesh Networks
WSN | Wireless Sensors Networks
WT | Wireless Terminal

During the last decade, Machine Learning (ML) algorithms, especially deep
learning (DL), have shown an outstanding performance in almost all fields such
as computer vision [1], image processing [2], natural language processing
(NLP) [3], etc [4]. This increased success encouraged the researchers to focus
on unleashing the power of ML for various problems and domains.
Unsurprisingly, ML was continuously breaking new records in all of these
fields. Different success stories have been recorded for ML, from AlexNet, a
Convolution Neural Network (CNN) model which won the ImageNet competition [5],
to AlphaGo, a reinforcement learning model introduced by DeepMind in 2016 [6],
and many other ground-breaking models.

The area of wireless communications has been heavily affected by recent
advances in ML algorithms and technologies. Specifically, ML was a main player
in the 5G and beyond communication systems. It has been adopted to provide
modern communication systems with a plethora of intelligent services and
functions such as intelligent resource allocation [7, 8], link adaptation [9,
10], beamforming [11, 12], automatic modulation classification [13, 14],
channel estimation [15, 16], or CSI compression [17, 18, 19]. Recently, the
advances in ML/DL techniques have also affected the design of medium access
control (MAC) protocols [20]. Designing an efficient MAC technique is a
complex and challenging task since it counts for many factors such as
collision, throughput, fairness, hidden terminal problems, etc. With the
accelerated developments in 5G towards the 6G, it becomes essential to develop
MAC protocols that support different applications scenarios such as ultra-
reliable low-latency communications (URLLC) [21, 22], enhanced wide broadband
[23] and massive machine-type communications [24]. Therefore, ML-inspired
techniques have great potential for improving and extending the existing MAC
mechanisms. As a result, it became an active area of research that attracted
many researchers from both academia and industry.

Figure 1: A trend in the number of publications in ML-based MAC protocols, as
reported in the Scopus database.

The rising interest in ML-based MAC techniques can be clearly observed by the
number of articles in this field. The results shown in Fig. 1 were returned
when searching the Scopus database with the query text ”Machine learning for
medium access techniques”. The figure plots the number of documents per year
returned by the query. We can see an exponentially growing trend in the number
of articles. Therefore, surveying and classifying the recent work in this area
along with a comprehensive review of the widely adopted ML/DL algorithms as
well as the MAC design issues can significantly help the researchers
accelerate their work.

Different survey papers for MAC protocols have been proposed to summarize the
various and latest directions. The survey in [25] reviewed and categorized
different MAC protocols depending on their medium access strategy (e.g.,
resource allocation, random access, etc). The authors in [26] reviewed various
MAC protocols according to the problem being addressed. The authors in [27]
surveyed the work published in the period from 2002 to 2011 in the area of MAC
protocols for WSN. Table. I shows a comparison between the different surveys
and tutorials in this area.

The following are the main points that distinguish this work from the existing
surveys:

  * •

We survey the recent work that has been published during a very active period
of time (i.e., 2012-2022).

  * •

We provide a focus on ML-based MAC techniques only. This makes this survey
more specialized for the impact of ML techniques on the problem of medium
access. This is an advantage for those who are interested not in the general
problem of medium access, but rather in the application of machine learning in
this specific problem.

  * •

Unlike prior work that focused on classifying the surveyed work from one
dimension, we provide a multidimensional classification such that the reader
can easily reach the work by their year, the adopted ML technology, or the
application domain.

  * •

For the sake of completeness, we comprehensively explain the various MAC
technique, their design issues, and the relevant ML algorithms. This puts
together all the pieces of the game in one place, which facilitates the
reading of the survey and the reader does not need to look for other resources
that explain the basic concepts and terminologies.

Readers can find that survey interesting and can stimulate future research
directions and problems on the topic of ML-based MAC techniques.

TABLE I: Comparison between other surveys in the literature and the proposed surveys | Publication Year | Survey Topic | Employed Technology | Application Domain | Time Period Covered
---|---|---|---|---|---
Huang et al. [28] | 2012 | MAC Protocols | General | WSN | 2002-2011
Binti et al. [27] | 2017 | MAC Protocols | Machine Learning | WSN | Unspecified
Isolani et al. [29] | 2018 | MAC | General | Wireless Networks | Unspecified
Quintero et al. [30] | 2018 | MAC | General | WSN | Unspecified
Zhang et al. [31] | 2019 | General | Deep Learning | Wireless Networks | Unspecified
Sun et al. [32] | 2019 | General | Machine Learning | Wireless Networks | Unspecified
Fatima et al. [33] |  |  |  |  |
Ours | 2020 | MAC | Machine Learning | Wireless Networks | 2011-2021

###  I-A Navigation: How you can read this paper?

Figure 2: The layout of the paper.

Navigating through this survey has many forms depending on the objective of
the reader. For those who are interested in gaining a piece of knowledge on
the underlying concepts and terminologies of ML, they can start with section
II. For the readers who are not familiar with the MAC techniques, III and IV
sections give a good background and place the reader in the context of MAC
technique design. The surveyed work is organized in sections V, VI, VII, VIII,
and IX. Fig. 2 provides the whole layout of this work. With the help of Fig.2,
the reader can easily navigate to the various sections depending on his/her
own interest. Moreover, Table II and Table III list the surveyed work along
with the objective of the work, adopted learning category, and application
domain.

##  II Machine learning Algorithms

Figure 3: Different algorithms under the umbrella of supervised learning. The
figure will include the supervised, unsupervised, and reinforcement learning
branches.

This section covers the various ML algorithms employed in the MAC problem. The
employed algorithms belong to one of the three main branches of ML, namely,
supervised, unsupervised, and reinforcement learning. We divide this section
into three subsections, one for each of the main branches:

Figure 4: A non-linearly separable data in the input space can be linearly
separable in a higher dimensional space.

###  II-A Supervised Learning

Supervised learning is the learning paradigm in which a model learns by
examples. This involves showing the model many solved examples of the same
problem in the form of (input, output) pairs, i.e. labeled datasets. The model
then learns how to map its input to the desired output. Based on the possible
values of the output can take, supervised learning can be divided into
classification models (discrete output), and regression models (continuous
output) [34]. Under each of the classification and regression categories,
there are many algorithms as shown in Fig. 3.

Among these algorithms, we will cover three of the most adopted algorithms,
namely, K-nearest neighbors, support vector machines [35, 36, 37], and neural
networks:

####  II-A1 K-Nearest Neighbours

K-nearest neighbors (KNN) is one of the simplest and most appealing non-
parametric models for both classification and regression. The intuition behind
KNN is that close data points usually share the same label [38, 39, 40]. The
closeness of data points is computed using a similarity measurement function.

$d(x,y)=\sqrt{\sum_{i=0}^{n}(x_{i}-y_{i})^{2}}.$ (1)

The performance of the KNN algorithm is largely affected by many two factors:
the considered number of nearest points, K, and the used similarity metric.
Euclidean distance (1) and Cosine similarity (2) are two widely used functions
with KNN.

$d(x,y)=\frac{x^{T}y}{||x||\;||y||}.$ (2)

Given the $K$ nearest point, the label of the query point is inferred by an
aggregation technique, such as majority voting or distance-weighted voting. In
majority voting, the $K$ nearest points assign the query point to the majority
class. This implies equally weighting all the $K$ points. In distance-weighted
voting, each point participates in the final decision based on its distance
from the query point [41]. The main concerns about KNN are that the execution
time increases with the dataset size [42], and the high sensitivity for
outliers, see Fig. 5.

####  II-A2 Support Vector Machines (SVM)

Figure 5: An overview of the KNN algorithm for classifying unknown points
(left). An overview of the SVM classifier (right).

Support Vector Machines (SVM) belong to the family of maximum marginal
classifiers. In this family, the model classifies between $N$-dimensional data
points by finding an $(N-1)$-hyperplane to split the classes while maximizing
the margins between the two classes. In the simplest form, SVM can be seen as
a linear classifier. However, in most cases, the data points are not linearly
separable. Therefore, they can be mapped to a higher-dimensional space using a
kernel function. The idea behind this mapping is that a dataset that is not
linearly separable in its original space, maybe linearly separable in higher
dimensional space as shown in Fig. 4 and Fig. 5. For more details on SVM, we
refer to [43].

####  II-A3 Artificial Neural Networks

Artificial neural networks (ANN), or simply neural networks (NN), is a form of
a supervised ML model that can be utilized for both classification and
regression problems. The basic building block of an ANN is called a neuron, or
a node. A set of neurons associated with certain inputs is called a layer. The
overall neural architecture may consist of one or more layers. Each input to
the node is associated with a weight value. The node then computes its output
by summing over all weights, the multiplication of each input with its
associated weight. An activation function is then applied to the result of the
summation as shown in (3).

$f(x)=\vartheta\left(\sum_{i=1}^{N}w_{i}\times x_{i}\right),$ (3)

where $N$ is the dimension of the input vector. The activation function,
$\vartheta$, takes many forms. Sigmoid, $f(x)=1/1+e^{-x}$, ReLU, $\max(0,x)$,
Tanh, and $f(x)=\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}}$, are all examples of widely
adopted activation functions.

The whole architecture is then trained to minimize a certain loss function.
The loss is minimized by updating the weights using an optimization algorithm
such as gradient descent, stochastic gradient descent, RMSProp, Adam, etc.

###  II-B Unsupervised Learning

While in supervised learning, a model learns by example from a labeled
dataset, in unsupervised learning, a model learns the underlying structure of
an unlabeled dataset. This branch of ML addresses two main problems, namely
clustering and dimensionality reduction.

As the name implies, clustering algorithms explore the similar underlying
structure of a dataset with the aim of grouping similar data points into one
group (i.e., cluster). $K$-Means [44], DBSCAN [45], and expectation-
maximization clustering using Gaussian mixture models (GMM) [46], are among
the most widely used clustering algorithms.

By contrast, dimensionality reduction is concerned with structuring the data
in a way that represents it in a lower-dimensional space as efficiently as
possible. Examples of the widely used algorithms for dimensionality reductions
are principle component analysis (PCA) [47], singular value decomposition
(SVD) [48], and linear discriminant analysis (LDA) [49].

Following are brief descriptions of the principal component analysis and
K-Means clustering algorithms that are often used to address each of the
aforementioned problems:

####  II-B1 $K$-Means Clustering

It is an algorithm that aims to partition a number $N$ of data points, into a
number $K$ of clusters [47]. Every data point is assigned to a specific
cluster With the nearest mean, cluster center, or cluster’s centroid. Based on
this interpretation, each cluster’s center acts as a prototype for its cluster
[44]. The algorithm then tries to find the best assignment for the data points
that minimizes the intra-cluster differences, given in (4).

$\sum_{j=1}^{K}\sum_{\forall x_{i}\in c_{j}}\left\|x_{i}-\mu_{j}\right\|.$ (4)

In real-world problems, $K$-means has been shown to be effective, but its
performance is highly dependent on how the cluster centroids are initialized,
as well as the hyperparameter, $K$.

####  II-B2 Principal Component Analysis (PCA)

Figure 6: The first principal component is the direction that captures the
highest variance between the projected data.

High-dimensional data is more time-consuming and can affect model learning.
Therefore, many ML algorithms and models have problems dealing with high-
dimensional data, i.e., the curse of dimensionality [50]. In these cases, we
would like to find the best representation of the data in a lower-dimension
space. Among the most widely used, and well-studied, approaches for this
objective is the principle component analysis (PCA) [51]. PCA can be seen as
the process of computing some principal components, some or all of which can
be used to change the basis of the data.

Using the projected data, the first principal component can be seen as the
direction capturing the most variance. The second principal component is the
direction that captures the second-highest variance for the projected data,
etc. A main limitation of PCA is that it can capture linear correlations
between the features. If the assumption is not satisfied, PCA can no longer be
applied in its current form and further modifications of the algorithm are
necessary, see Fig. 6. For more details about PCA and its analysis, we
recommend reading [52].

###  II-C Reinforcement Learning (RL)

Reinforcement learning (RL) is a new paradigm of ML which is different from
supervised and unsupervised learning. Supervised and unsupervised ML
algorithms find insights in the pre-collected datasets. Specifically, the
supervised ML maps the input samples to the corresponding label, while
unsupervised ML finds the pattern in this data. On the other hand, RL targets
sequential decision-making tasks. RL builds a strategy that helps an
intelligent agent to take actions based on the perceived environment. which
mimics the human learning process that learns through trial and error [53].

In RL, there is no learning from a dataset, only a real number or reward
signal. The decision-making is done sequentially, and hence time notation is
essential in the problem formulation [54]. RL agent, and depending on
interaction with the surrounding environment, takes sequential actions. At
each time step, $t$, the agent selects an action, $a_{t}$, from a set of all
possible actions, $A$, which is called the actions space. The environment
receives the action, $a_{t}$, and returns the corresponding reward as well as
the next state. The reward signals how well an agent is doing in a given state
by letting it know how successful or unsuccessful its actions are. The RL
algorithms can be classified into the following classes:
1) Value-Based: where the agent tries to maximize a value function, $V(s)$,
which is the expected long-term return of a state under a particular policy,
as opposed to the short-term reward.
2) Policy-based: where the agent searches for a rule or policy to take any
action in every state to gain maximum reward in the future. The policy is the
mapping that the agent learns to determine the next action based on the
current state.
3) Model-Based: where the environment is modeled, and the agent learns to take
actions based on that specific model.

##  III Taxonomy of Medium Access Mechanisms

Figure 7: A taxonomy for various medium access mechanisms.

Several MAC protocols have been proposed in the literature. According to [33],
these protocols can be categorized into contention-based and contention-free
protocols, see Fig. 7. In this section, we briefly introduce each of these
categories and the protocols under each of them.

###  III-A Contention-Based

A content-based protocol is considered the simplest protocol in terms of setup
and implementation. Typically, multiple users can simultaneously communicate
on a single radio channel using a contention-based protocol [55]. In this
case, several nodes contend to gain access to a shared channel. The main
limitation of these protocols is the lack of scalability. Specifically, the
number of collisions increases with the number of nodes. The basic contention-
based protocols are as follows:

####  III-A1 ALOHA

ALOHA protocols try to share channel bandwidth by using extra brute force
manners. The first ALOHA protocol was part of the ALOHAnet project at the
University of Hawaii when it was developed [56]. Strangely, the lack of
channel access control is one of the main features of ALOHA. When a node needs
to transmit a packet, it is allowed to be transmitted immediately. In such a
system, collisions are usually common. A feedback mechanism, i.e. automatic
repeat request (ARQ), is required to ensure packet delivery. If a node finds
that its packet was not successfully delivered, a schedule for the packet to
be re-transmitted will be done. Of course, because of packet vulnerability,
ALOHA channel utilization is poor. The results presented in [57] show that to
improve protocol performance dramatically, a synchronous communication model
can be used. Every node had been forced to wait, by slotted ALOHA, for the
start of a slot for transmitting its packet. This way the amount of time a
packet is susceptible to collision is reduced, and ALOHA channel utilization
doubles effectively.

####  III-A2 Carrier Sensing Multiple Access (CSMA)

Several MAC protocols used a carrier detection strategy to detect collisions
that may happen with ongoing transmissions. First, these protocols listen to
the channel to see if there is an ongoing activity. If the channel is free, it
will initiate a packet transmission, and if a channel is busy it will suppress
it [58]. Persistent CSMA continuously listens, while the channel is busy, to
determine when activity stops. The protocol transmits a packet immediately
when the channel returns to an idle state. When multiple nodes are waiting for
a free channel, Collisions could occur. Randomization had been introduced by
Non-persistent CSMA, which will reduce the probability of such collisions.
Simply, a source node waits a random backoff time, whenever a busy channel is
detected, before retesting the channel. With an exponentially increasing
random interval, this process is repeated, until the channel is found clear.
The $p$-persistent CSMA protocol represents a mid-point between persistent and
non-persistent CSMA. For that case, it is accepted to make the channel
slotted, and not to consider the time synchronized. The carrier detection
occurs at the beginning of each slot and every slot length is worth the
highest propagation delay. A packet with probability $p$ such that $0<p<1$ is
transmitted by the node if the channel is free. This procedure will be
executed until either the channel becomes busy, or the packet is sent. A
source node will be forced to wait a random amount of time, by a busy channel,
before starting the procedure again.

Two other well-known variations of CSMA are the Carrier Sensing Multiple
Access with Collision Detection (CSMA/CD) and the Carrier Sensing Multiple
Access with Collision Avoidance (CSMA/CA). In CSMA/CD, the node continues
sensing the channel to detect possible collisions from other nodes’
transmissions. CSMA/CA uses a handshaking dialog to minimize the number of
exposed nodes and reduce hidden node interference. The mentioned handshake is
a control packet of Request-to-Send (RTS) sent to its destination from a
source node. A control packet clear-to-send (CTS) is the response of the
destination, which completes the required handshake. The response from CTS
permits the source node to transmit the packet it has. A node is forced to
reschedule for later transmission of the packet because of the lack of a CTS
[59].

###  III-B Contention-Free

In contention-free protocols, each node does not have to contend to transmit
its data. In the following subsection, we elaborate on the contention-free
resource allocation category:

####  III-B1 Resource Allocation

In resource allocation-based protocols, each node is assigned a virtual
subchannel to transmit its data through, see Fig. 8. These virtual subchannels
can be time, frequency, or code. Based on the type of the assigned subchannel,
we can divide this class into three sub-classes:

##### Time Division Multiple Access (TDMA)

TDMA divides the total bandwidth of the channel into $M$ slots of equal time.
They are organized into a synchronous frame. Conceptually, every slot
represents a channel its capacity equals $C/M$ bps. The total channel
bandwidth capacity is $C$ bps. One, or more, time slots can be allocated for
every node to be exclusively used. Thus, for a TDMA system, packet
transmission happens in a serial fashion, and the accessing of the channel
happens in turns for every node. The time of $L/C$ is required for
transmitting the $L$-bit packet since every node has access to the whole
channel bandwidth in every time slot. However, for the same node, a delay of
$M-1$ slots between consecutive packets will happen in case every node is
allocated one slot per frame exactly.

##### Frequency Division Multiple Access (FDMA)

To keep interference from happening to the co-channel, FDMA divides the total
bandwidth of the channel into equal sub-channels, that are separated through
guard bands. $C/M$ is every sub-channel capacity and $C$ is the total channel
bandwidth allocated capacity, assuming that the little amount of frequency
that had been lost to the guard bands is neglected. One, or more, of these
sub-channels, could be assigned for each source node for its own exclusive
use. The main advantage of FDMA is the capacity to allow concurrent packet
transfers, one on every sub-channel, with no collision. This increased
flexibility comes with the cost of increased packet transmission time, which
results in a bigger packet delay.

##### Code Division Multiple Access (CDMA)

CDMA allows transmissions to occupy the channel simultaneously without
interference, while FDMA and TDMA isolate transmissions into different
frequencies or times. Using special coding methods, that make it possible to
retrieve the information out of combined signal, collisions are avoided. If
two nodes have enough orthogonal codes, i.e. different codes, their
interference with each other during transmissions will not happen. CDMA
functions by successfully spreading the bits of information over an
artificially widened channel [60]. That way, the diversity of frequency for
every transmission is increased, which makes it susceptible less to fading and
making the interference level reduced which could happen to other systems that
operate in a similar spectrum. Thereby, the design of the system and its
deployment are simplified as every node shares a similar frequency band.
However, highly sophisticated, expensive, and typically more difficult-to-
manage hardware are required for CDMA systems.

Figure 8: Different types of resource allocation-based techniques.

##  IV MAC Protocol Design Issues

Many protocols have already been proposed for solving the medium access
problem in wireless networks [35]. When designing a MAC protocol, there are
important issues that should be considered. These issues vary from one
application to another. For example, in VANETs, the nodes have high-speed
mobility, resulting in frequent and fast changes in the topology. While in
WiFi networks, the nodes have less mobility and significantly lower speed,
which concludes in slower and less frequent topology changes. In this section,
we list some of the important issues that should be considered when designing
a MAC protocol:

  1. 1.

Bandwidth Efficiency: Due to the limited resource of the radio spectrum, the
bandwidth is limited. To use this bandwidth efficiently, different techniques
for decreasing the control overhead should be adopted. Bandwidth efficiency
could be defined as the bandwidth ratio used for the transmission of the data
to the overall bandwidth available. Increasing this number means we use more
bandwidth ratio for data transmission, which is favorable for efficient
bandwidth utilization.

  2. 2.

Quality-of-Service Support: Voice and video communications, among many other
applications, require a certain quality of service (QoS) that must be
guaranteed by the network operator to ensure proper functionality. Reservation
of bandwidth is one way to maintain good QoS. However, since nodes in some
networks could be mobile, this solution can be difficult to handle in MANETS,
for example.

  3. 3.

Synchronization: In wireless networks, it is of crucial importance to
synchronize the operation of different nodes. Reaching this sort of
synchronization in centralized networks is easier due to the existence of a
centralized clock, i.e., a centralized clock shared by the infrastructure.
However, the synchronization is not so straightforward in cases where the
control is completely distributed.

  4. 4.

Hidden and Exposed Terminal Problem: It is easy to understand the problem of
the hidden terminal but not easy to resolve it. Three wireless nodes: A, B,
and C are shown in Fig. 9 (a). Since A and C are not within communication
range of each other, they cannot communicate directly through their physical
layer. But both can simultaneously communicate with B, and B exists in their
range of communication. Suppose B receives data from A. Simultaneously at this
point, this transmission could not be heard by C. The transmission by C can
happen at any time, which can result in node B having a transmission collision
during the transmission that is ongoing with node A. This is the problem of
the hidden terminal, where A and C together are not heard from each other. The
exposed terminal problem is like the problem of the hidden terminal. The cause
of the problem is the coverage limitation of communication within the area and
shared medium. In Fig. 9 (b) the problem is explained and shows four wireless
nodes: A, B, C, and D. When B sends data to A, C is forced not to send to D
because it has confidence that it will disrupt the data transmission happening
from node B to node A. But D can receive from C with no risk of disrupting
data transmission happening between A and B. C is the exposed node.

  5. 5.

Error-Prone Shared Broadcast Channel: When a sender node transmits to a source
node, no other node in the source node neighborhood should be transmitting due
to the broadcast radio nature; otherwise, a collision may occur. Because all
nodes that exist close to a transmitted node can hear the transmission of that
node. It is often happening that the medium is accessed at the same time by
nodes in the same neighborhood, therefore the probability of transmission
collisions in wireless networks is quite high. The role of a MAC protocol is
to reduce these communication collisions as much as possible.

  6. 6.

Distributed Nature and No Central Coordination: An ad hoc network’s key
characteristic is the insufficiency of centralized coordination or centralized
infrastructure. Nodes should communicate in an absolutely distributed manner,
in such ad hoc networks. The MAC protocol should perform efficient channel
access control depending upon a number of control packet exchanges that may
reduce bandwidth.

  7. 7.

Mobility of Nodes:  Assuming stationary nodes happen, scheduling the channel
access would be much easier than with mobile nodes. A MAC protocol should
consider the nature of mobility and the speed of nodes to provide the most
efficient mechanism [61].

Figure 9: The hidden node problem (a) and the exposed terminal problem (b).

TABLE II:

| Objective | Employed Technique | Application Domain
---|---|---|---
|  Optimize Power Consumption |  Maximizing Throughput |  Maximizing Channel/Spectrum utility |  reduce control overhead |  Security (Wireless Intrusion Detection) |  Supporting real time communications |  Maximize Video streaming |  Speed up the learning process for Q-learning algorithms |  Reduce implementation cost |  Frame-based procedure |  MAC Selection for QoS requirements/dynamic circumstances |  Reinforcement Learning |  Deep Reinforcement Learning |  Machine Learning based MAC |  Heterogeneous IoT Networks |  Body Sensor Networks |  M2M Networks |  Cognitive Radio |  WSN |  Mobile wireless applications |  Real-Time Wireless Networks |  Mobile Ad-hoc networks |  Wireless mesh networks |  Wireless network virtualization |  Cognitive Radio ad-hoc networks
Wang et al. [62] | ✓ |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Hu et al. [63] |  | ✓ |  |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |  |  |  |  |  |  |  |  |
Shah et al. [64] |  |  |  |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |  | ✓ |  |  |  |  |  |  |
Hu et al. [37] | ✓ |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |  |
Gilani et al. [65] | ✓ | ✓ |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |
Shah et al. [66] |  |  |  |  |  |  |  |  |  |  | ✓ |  |  | ✓ |  |  |  |  | ✓ |  |  |  |  |  |
Liu et al. [67] |  | ✓ |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |  | ✓ |  |  |  |  |  |  |  |
Liu et al. [68] | ✓ |  | ✓ |  |  |  |  |  |  | ✓ |  |  |  |  |  |  | ✓ |  |  |  |  |  |  |  |
Alvi et al. [69] | ✓ |  |  | ✓ |  |  |  |  |  | ✓ |  |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |
Alvi et al. [70] | ✓ |  | ✓ | ✓ |  |  |  |  |  | ✓ |  |  |  |  |  |  |  |  |  ✓for Smart-Cities |  |  |  |  |  |
Qiao et al. [71] |  |  |  |  |  |  |  |  |  |  | ✓ |  |  | ✓ |  |  |  | ✓ |  |  |  |  |  |  |
Ngo et al. [72] |  | ✓ |  |  |  |  |  |  |  | ✓ |  |  |  |  |  | ✓ |  |  |  |  |  |  |  |  |
Ford et al. [73] |  |  |  |  |  |  |  |  |  |  |  |  |  | ✓ |  |  |  | ✓ |  |  |  |  |  |  |
Yu et al. [74] |  |  |  |  |  |  |  |  |  |  |  |  | ✓ |  |  |  |  |  | ✓ |  |  |  |  |  |
Abdulhammed et al. [75] |  |  |  |  | ✓ |  |  |  |  |  |  |  |  | ✓ |  |  |  |  | ✓ |  |  |  |  |  |
Shoaei et al. [76] |  |  | ✓ |  |  |  |  |  |  |  |  | ✓ |  |  |  |  | ✓ |  |  |  |  |  |  |  |
Yang et al. [77] |  |  |  |  |  |  |  |  |  |  |  |  |  | ✓ | ✓ |  |  |  |  |  |  |  |  |  |
Derakhshani et al. [78] |  |  |  | ✓ |  |  |  |  |  | ✓ | ✓ |  |  |  | ✓ |  |  |  |  |  |  |  |  |  |
Chilmulwar et al. [79] | ✓ | ✓ |  |  |  |  |  |  |  |  |  |  |  | ✓ |  |  |  |  |  | ✓ |  |  |  |  |
Franchino et al. [80] | ✓ |  |  |  |  | ✓ |  |  |  |  |  |  |  |  |  |  |  |  |  |  | ✓ |  |  |  |
Cheng et al. [81] |  |  | ✓ |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |
Ju et al. [82] |  |  |  | ✓ |  |  |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |  |  | ✓ |  |  |
Naddafzadeh et al. [83] |  | ✓ |  |  |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |  | ✓ |  |  |  |  |  |
Mihaylov et al. [84] | ✓ | ✓ |  |  |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |  | ✓ |  |  |  |  |  |
Chu et al. [85] |  |  |  | ✓ |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |  | ✓ |  |  |  |  |  |
Changuel et al. [86] |  |  |  |  |  |  | ✓ |  |  |  |  | ✓ |  |  |  |  |  |  | ✓ |  |  |  |  |  |
Galzarano et al. [87] | ✓ |  |  |  |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |  | ✓ |  |  |  |  |  |
Hu et al. [36] |  |  | ✓ |  |  |  |  |  |  |  |  |  |  | ✓ |  |  |  | ✓ |  |  |  |  |  |  |
Rovcanin et al. [88] |  |  |  |  |  | ✓ |  |  |  |  |  | ✓ |  |  |  |  |  |  | ✓ |  |  |  |  |  |
Al et al. [89] |  | ✓ |  |  |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |  |  |  |  |  | ✓ |  |
Chu et al. [90] | ✓ | ✓ |  |  |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |  | ✓ |  |  |  |  |  |
Amuru et al. [91] |  |  |  |  |  |  |  | ✓ |  |  |  | ✓ |  |  |  |  |  |  | ✓ |  |  |  |  |  |
Shoaei et al. [92] |  |  | ✓ |  |  |  |  |  | ✓ |  | ✓ |  |  |  |  |  |  |  |  |  |  |  |  | ✓ |
Kakalou et al. [93] |  | ✓ |  |  |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |  |  |  |  |  |  |  | ✓
Mastronarde et al. [94] | ✓ |  |  |  |  |  |  |  |  |  |  | ✓ |  |  |  |  |  |  |  | ✓ |  |  |  |  |
Qiao et al. [95] |  |  | ✓ |  |  |  |  |  |  |  |  |  |  | ✓ |  |  |  |  | ✓ |  |  |  |  |  |
Cordeiro [96] | ✓ | ✓ | ✓ |  |  |  |  |  |  |  | ✓ |  |  | ✓ |  |  |  |  | ✓ |  |  |  |  |  |
Xu et al. [97] |  |  | ✓ |  |  |  |  |  |  |  |  |  | ✓ |  |  |  |  |  | ✓ |  |  |  |  |  |

TABLE III: Taxonomy of MAC protocols in the last decade Year | Ref. | Taxonomy | Description
---|---|---|---
2010 | [62] | HMAC | Hybrid MAC
2011 | [63] | LA-MAC | Load Adaptive - MAC
2011 | [64] | DDHMAC | Dynamic-Decentralized Hybrid MAC
2013 | [66] | SAML | Self-Adapting MAC
2015 | [69] | BSMAC | Bitmap-Assisted Shortest Job MAC
2016 | [70] | BEST-MAC | Bitmap-Assisted Efficient & Scalable TDMA based MAC
2017 | [72] | TSCH | Time Slotted Channel Hopping
2019 | [74] | DLMA | Deep Learning MAC

##  V Wireless Sensor Networks (WSN)

A WSN is a network that consists of many nodes, these nodes can sense and
process data through multi-hop communication. Actually, WSN nodes have
different transmission behavior and variations in their traffic loads, see
Fig. 10. The WSN lifetime mostly depends on the efficiency of energy due to
the difficulties of replacing or recharging nodes [98]. Other challenges are
facing WSNs according to the used MAC protocol, such as increased delay and
less throughput [99]. A MAC protocol designed with good quality is required to
make sure of successfully delivering the data while keeping the energy
consumption at its minimum.

Figure 10: A wireless sensor network (WSN) fusing the sensed data to a control
facility through the Internet.

In [84], WSNs create a derived collection, i.e. class of networks. Our daily
environment could be monitored by this derived collection of networks.
Typically, a WSN consists of a set of sensor nodes with limited resources in
terms of energy and communication. These limited resources complicate a WSN
application design where the application requirements (e.g., throughput,
latency, or lifetime) often conflict with the network capacity and the energy
resources. To address these challenges, a standard approach is followed. This
standard approach depends on scheduling wake-up that alternates between sleep
and active states for sensor nodes. On-demand paging, synchronous, and
asynchronous wake-up are the three commonly used wake-up solutions.

  * •

With on-demand paging, a separate radio managed the wake-up function, and much
less power is used by the separate radio than the master radio when inactive.
Therefore, the master radio remains in a state of sleep until a signal came
from the slave radio tells that the radio channel is about to receive a
message. However, there is an additional effort in hardware design that should
not be neglected.

  * •

In the synchronous wake-up approaches, the nodes turn on their radios in a
coordinated manner, allowing nodes to wake up at predetermined times when
communication between nodes becomes possible. The main problem with such a
solution is the overhead caused by synchronizing the nodes.

  * •

Finally, with asynchronous wake-up solutions, the nodes are unaware of each
other’s schedules, and the communication involves increased costs for both
sender and receiver.

The authors in [84] presented a self-organizing approach using RL to schedule
the node’s wake-up cycles in a WSN. Two main conditions must be met for nodes
to wake up.

  1. 1.

For communication to succeed, the node’s parent in the routing tree needs to
be awake.

  2. 2.

The neighboring nodes should not be active except for the parent node.

Also, the schedules of the neighboring nodes that do not need to communicate
are desynchronized to avoid radio interference and packet loss. Nodes in the
WSN use an RL algorithm to learn a wake-up schedule, i.e. within the same
frame when to remain active, such that in a distributed environment lifetime
and throughput will be improved. Each node stores a quality value (Q-value)
for each slot within its frame. This value indicates how advantageous for the
node to stay awake during these slots for each frame, i.e. given the work
cycle of the node and considering node communication history, which is an
efficient wake-up pattern. When a communication event occurs at a node, i.e. a
packet is listened to, received, or sent, or when at the time of active period
events have not occurred, i.e. inactive listening, the Q-value of the slot(s)
will be updated by that node. Experiments are conducted on three network
topologies namely line, mesh, and grid topologies with different sizes.
Experimental results show that both synchronous and asynchronous are crucial
in WSN applications that demand real-time data. The method proposed in [84]
allowed the coordination of agents in their wake-up cycles with no
communication overhead and make sure that neighboring nodes evade interference
of communication at no energy cost. To decrease the number of time slots that
are active inside a frame and to relax the condition that the nodes’ active
time in a period that is contiguous, many optimizations are needed.

TABLE IV: BS-MAC and BMA-RR based MAC protocol transmission delay of data
comparison.

Node |  Data length (bytes) |  Data rate (bps) |  Time to send data (ms) |  Bits/slot in BMA-RR |  Slot length BMA-RR MAC (ms) |  Req slots |  Time to send data in BMA-RR (ms) |  Bits/slot in BS-MAC |  Slot length in BS-MAC |  Slots required |  Time to send data in BS-MAC (ms) |  Time lapsed in BMA-RR (ms) |  Time lapsed in BS-MAC (ms)
---|---|---|---|---|---|---|---|---|---|---|---|---|---
A |  120 | 24,000 | 40 | 2,000 | 83.33 | 1 | 83.33 | 200 | 8.33 | 5 | 41.67 | 43.33 | 1.67
B |  180 | 24,000 | 60 | 2,000 | 83.33 | 1 | 83.33 | 200 | 8.33 | 8 | 66.67 | 23.33 | 6.67
C |  210 | 24,000 | 70 | 2,000 | 83.33 | 1 | 83.33 | 200 | 8.33 | 9 | 75.00 | 13.33 | 5.00
D |  240 | 24,000 | 80 | 2,000 | 83.33 | 1 | 83.33 | 200 | 8.33 | 10 | 83.33 | 3.33 | 3.33
E |  280 | 24,000 | 93.33 | 2,000 | 83.33 | 2 | 166.67 | 200 | 8.33 | 12 | 100.0 | 73.33 | 6.67

In [87], a WSN typically consists of small, inexpensive, and low-power devices
that provide data acquisition, processing, and wireless communication
functions. Network endurance is a characteristic of WSN which is considered
very important. Sensor nodes can be deployed in remote areas such that
replacing or maintaining batteries is a difficult or impossible task. Access
to the shared communication channel is the responsibility of the MAC protocol,
also it is intended to ensure that successful radio management happens with
efficient energy management. Efficient energy management can be achieved by
avoiding the sources that cause waste of energy, i.e. Overhearing, inactive
listening, packet collisions, and excessive retransmissions. Adaptive behavior
that takes into account the actual network conditions is urgently needed to
address the network lifetime issue. This research introduces QL-MAC, which is
a new contention-based protocol for MAC belonging to WSN. The proposed
protocol employs the Q-learning algorithm to get an effective wake-up strategy
to decrease the consumption of energy based on the real load of the network of
neighborhoods. It gains from the interaction of the cross-layer with the
network layer to provide an improved understanding of the patterns of
communication and hence decrease the consumption of energy because of
overhearing and inactive listening. The goal of the protocol proposed is to
let the behaviors of nodes be inferred from each other to approve a virtuous
active/sleep schedule policy. Typically, RL is a machine learning sub-area
specialized in how a user takes actions to increase some reward that will be
gained in the long term. Q Learning is the most popular and powerful algorithm
based on RL and it does not require the modeling of the environment and its
actions depending on a Q function that determines a certain action quality at
a certain state of an agent. In particular, each node has to decide whether it
should be in active or in sleep mode during each single time slot. All nodes
store a set of Q values, that exist in an explicit slot inside the frame. The
Q value provides evidence of the gains a node has when during the relevant
time slot, the node is awake. The Q value is modernized over time based on
some explicit events that happen through the very slot in every frame.
Simulations are conducted using OMNET++ to compare the performance of the
proposed protocol against related MAC protocols in terms of Packet Delivery
Ratio (PDR) and nodes’ average consumption of energy. Results show that QL MAC
outperforms related MAC protocols as the proposed protocol allows nodes to
spend much less energy due to the sleep/wake-up radio schedule. Also, the
results show that when the slots number decrease, QL MAC generally exhibits
higher performance respectfully to the PDR but, the energy spent by the node
tends to increase.

A number of MAC protocols have been proposed for WSNs to ensure the successful
delivery of data while reducing energy consumption. The proposed protocols
have made a great impact on the efficiency of energy and the performance of a
channel. But, with the advances in research, the complexity of MAC protocols
is increasing and the overhead of the control packet consumes more resources
of a channel. That raised the need for simpler and more efficient MAC
protocols. MAC protocols for WSNs might be classified into different
categories such as TDMA-based [69, 70],[80], Hybrid MAC protocols where two
different protocols are combined together [62, 65], adaptive/dynamic MACs
[66], [95], [94], [76], [87], CSMA-based [65],[94], and ALOHA-based [85],[90].
The coming sub-sections A, B, C, and D, will explain these categories in more
detail.

###  V-A TDMA-Based MACs

In [69], the authors present an adaptive TDMA-based MAC protocol, called
Bitmap-assisted Shortest job first based MAC (BSMAC) for hierarchical wireless
sensor networks. The number of small-sized slots of time that BSMAC takes into
account is not the same as the number of member nodes number. The Shortest Job
First (SJF) approach is used to decrease the node average delay of a packet
and shorten the job time completion of the node. The proposed protocol
minimizes the control overhead and energy consumption by using 1 byte short
address to identify the member nodes. Experiments are conducted to evaluate
the proposed Hybrid MAC protocol against existing MAC protocols in terms of
throughput, energy efficiency, and delay. The results show that BSMAC
outperforms the existing MAC protocols in terms of throughput. Additionally,
BSMAC uses less energy than the rest of the protocols of MAC for the
equivalent quantity of data. BSMAC has significantly less transmission delay
due to the implication of the SJF algorithm as nodes transmit their data at
once instead of transmitting in parts. Table IV compares the transmission
delay of the BS-MAC and BMA-RR based MAC protocol.

In [70] the authors present a new TDMA-based MAC protocol, called Bitmap-
assisted Efficient and Scalable TDMA-based MAC (BEST-MAC). The primary goal of
BEST-MAC is to enhance quality control in applications for smart cities where
diversified traffic is in demand and data loss, or delays are undesirable. In
literature, several TDMA-based MAC protocols have been proposed to overcome
certain E-TDMA and TDMA limitations at the expense of higher control
overheads. Furthermore, these techniques do not deal with scaling problems as
they offer a fixed amount of data slots that are the same number as the
participant nodes, which are unable to handle adaptive traffic load
efficiently. BEST-MAC uses a large number of small-size time slots for a
growing Link usage that efficiently adapts to changing traffic demands. The
algorithm of Knapsack is employed to shorten the time it takes for a node to
complete a task and allows an increase in the number of nodes that may
transfer data concurrently. The suggested scheme also adds a distinct conflict
access duration to allow non-member nodes to join the network while data is
being transferred. Control expenses and energy consumption are both minimized
by giving each member node a one-byte short address to pinpoint them.
Experiments are conducted to evaluate the BEST-MAC protocol against
conventional MAC protocols in terms of throughput, energy efficiency, and
delay. The results show that BEST-MAC significantly improves throughput
because of the use of the optimization approach of knapsack and the choice of
smaller slots of data. Also, results show that BEST-MAC has less transmission
delay and use of energy in comparison to the current protocols of MAC. In
literature, real-time communication and energy saving over wireless networks
have received much attention. However, addressing both problems simultaneously
is rarely considered.

In [80], the authors proposed an energy model to reduce energy consumption in
wireless networks with real-time requirements. In particular, the paper
presents El-MAC, an elastic energy-aware algorithm to save energy at the
communication level, precisely at the MAC level where each node can adapt its
bandwidth requirements to balance performance versus energy consumption. The
authors consider collision, overhearing, control packet overhead, and idle
listening as the main sources of energy waste. Also, the work considers the
energy consumed by a transceiver to switch between operating modes (active and
sleep) which could be greater than that needed to stay always active.
Experiments are conducted using two different radio transceivers to evaluate
the proposed scheme. In experiments, a theoretic upper bound on channel
utilization is used as a performance measure. The results show that the
proposed El-MAC protocol is directly applicable to TDMA scheduling approaches.

###  V-B Hybrid MACs

In [62], the authors investigated QoS-based routing in the MAC layer. The
paper exhibits a new cross-layer effective protocol of MAC, referred to as
Hybrid MAC (HMAC), acceptable for WSNs relating to latency, the efficiency of
energy, and design complexity. HMAC allows the awareness of compromise between
different performance measurements as it couples Time Division Multiple Access
(TDMA) based MAC protocols and channel allocation schemes from existing
contention-based. Compared with other TDMA-based MAC protocols, HMAC presents
an extremely low-cost solution as it uses a new wake-up scheme, every node
does not have to keep neighborhood information and a little slotted frame
structure. HMAC uses three measures to evaluate its performance:

  * •

Average energy consumption;

  * •

Average end-to-end delay;

  * •

Delivery ratio.

Experiments show that for energy consumption the nodes in HMAC consume less
energy and the remaining energies of the nodes in HMAC are distributed evenly
than the existing MAC protocols. For the end-to-end latency, HMAC
significantly reduces the end-to-end delivery latency and provides higher
delivery ratios due to the use of a little frame length that helps nodes to
fast move packets to their next hops and, hence, reduces many collisions and
queuing delays.

Figure 11: Enabled mode beacon for the structure of superframe.

In [65] the authors present a CSMA/TDMA Hybrid MAC protocol for improving
features of IEEE 802.15.4 standard in terms of energy consumption and
throughput. The proposed protocol overcomes the weakness caused by the CSMA/CA
method, utilized in IEEE 802.15.4 standard, in conditions like high loads. In
CSMA/CA, the collision avoidance mechanism is not efficient in the case of a
large-scale WSN. Also, the slotted CSMA/CA used in IEEE 802.15.4 causes little
network throughput because of the collisions that result from many concurrent
transmissions, at the beginning of a new superframe, see Fig. 11. Regarding
these issues, the work proposed in [65] allows the coordinator to suitably
divide the Contention Access Period (CAP) between slotted CSMA/CA and TDMA
relative to the collision level detected on the network and the data queue
state of nodes. Accurate information of queue state is determined using
reserved bits in the standard data packet header. Two main changes are applied
to standard IEEE 802.15.4, the first change is applied to add a TDMA period to
IEEE 802.15.4 standard using the beacon frame data fields and the other is
applied to enable the coordinator to change the border between CSMA and TDMA
in CAP. Experiments are conducted on OMNeT++. Experimental results of the
hybrid protocol are examined to IEEE 802.15.4 standard. Two different
conditions have been tested, namely (BO, SO)=(7, 6) and (BO, SO)=(3, 2). The
results show that the CSMA/TDMA hybrid protocol enhances the throughput of the
network by 1.6 and 2.3 times bigger than the standard IEEE 802.15.4. The
average energy consumption has been reduced by 38% and 70% for the two
conditions of (BO, SO)=(3, 2) and (BO, SO)=(7, 6), respectively.

###  V-C Adaptive/Dynamic MACs

Integrating wireless sensors with mobile phones confront environments with
dynamic nature where both requirements of the application and wireless
conditions environment are changing quickly. Existing MAC protocols cannot
provide optimal performance for such dynamic environments. Pursuing a one-MAC-
fits-all approach avoids applications from achieving the changing
environmental conditions, needs of different workloads, or Quality of Service
(QoS) requirements.

The authors in [66] present a Self-Adapting MAC Layer (SAML) which selects the
most appropriate MAC protocol for the current conditions and requirements.
SAML supports run-time switching among different MACs. As it is equipped with
a MAC Selection Engine that enhances the dimensions of latency, energy
consumption, and reliability, and considers dynamic external interference. The
engine includes three major modules, traffic monitor, noise monitor, and
classifier. The application traffic pattern had been kept on track by the
traffic monitor by calculating the mean and variance of Inter-packet Interval
(IPI) of the send commands called by the application. The noise monitor
calculates the external interference degree in the environment by calculating
the mean and variance of the Received Signal Strength (RSS). The Classifier is
finally used to determine the best MAC relative to the existing application-
specified REL order, i.e. Reliability (R), Energy consumption (E), and Latency
(L), and the values released from the Noise and Traffic Monitors. Experiments
are conducted on TinyOS 2.x. Experiments using three models containing a
maximum of five MACs. For evaluation, a new gateway device that integrates an
IEEE 802.15.4 radio with Android phones is used. The results show the
efficiency of SAML in avoiding memory bloat as well as its efficiency in
controlling the network nodes.

The work in [95] introduces a MAC protocol selection model for the dynamic
network environment. Various network parameters are chosen to construct the
feature data set to train a classification model. When a satisfying
classification accuracy is reached, the optimal MAC protocol could be selected
by the MAC selection module. A two-stage selection model is proposed. Stage
one is the classification learning process; obtaining an optimal
classification result using the Sequential Minimal Optimization (SMO)
algorithm which trained different types of feature datasets. Stage two is the
selection decision process; the network nodes, with the help of a classifier
model, choose an appropriate MAC protocol. The CSMA/CA and Dynamic TDMA are
the candidate MAC protocols. Simulations are conducted to evaluate the
performance of the selection module. The simulation results show that the
proposed model achieves a weighted average Probability of Correct
Classification (PoCC) of 94%. This is the best PoCC ratio compared to those
achieved by other benchmark models such as Naive Bayes, J48, and Random Forest
(RF). Moreover, the results show that the MAC protocol selection model had the
capability to select the appropriate MAC protocol that best fits the current
network environment.

In [94], the authors present a rate adaptive fully distributed CSMA/CA
protocol. By giving users reduced congestion windows, the proposed protocol
gives access probability to users who need to send critical data. It assists
users in minimizing their energy consumption depending on their limitations of
delay by collaborating with the rate-adaption algorithm at the physical layer.
In particular, a multi-user delay-sensitive energy-efficient scheduling
problem is formulated as a Markov Decision Process (MDP). An RL algorithm is
proposed so the single-user problems will be solved online. This solution
enables decreasing energy exhaustion to users relative to their delay
constraints, bear in mind that traffic, channel, and multi-user dynamics are
not known beforehand. Compared to the conventional CSMA/CA protocol, the rate-
adaptive CSMA/CA protocol proposed in [94] determines congestion windows (CWs)
to increase the probability of channel access for users who want higher
transmission rates. In that way, the MAC protocol serves the goal of
decreasing the consumption of energy for every user based on their delay
constraints when linked with the suggested algorithm of transmission
scheduling. Moreover, using this approach, when a user takes the channel,
other users stop their counters (backoff). Once the user frees the channel,
other users resume their counters assuming that other users reset their
counters (backoff) at the end of every time slot. Accordingly, a user’s
counter (backoff) will be updated to show its present state. Experiments are
conducted to evaluate the energy and delay performance of the proposed
protocol against the conventional CSMA/CA through MATLAB simulations. The
results show that the rate-adaptive CSMA/CA protocol allows all users to
adhere to their constraints and uses 65% less power than the traditional
protocol of CSMA/CA.

The authors in [76] present a learning-based MAC which is reconfigurable where
the division between contention-based and contention-free access regimes in
every frame is changed according to the status of the network. The logic
behind the suggested MAC protocol allocates devices that have a high
probability of transmitting packets to the disagreement-free regime, while the
remaining devices are permitted to engage in competition in the disagreement-
based regime. This scheduling is formulated as an optimization problem to
maximize the network throughput while maintaining the slice reservations. The
proposed protocol employs an RL algorithm based on Thompson Sampling (TS) to
collect arrival packet probabilities of devices. Based on learned
probabilities, an algorithm is developed in which the Deterministic Assignment
(DA) and Random Access (RA) partitions are determined by a threshold. Devices
with expected throughput higher than a certain threshold are acknowledged for
DA, while the remaining devices transmit in the RA regime. Experiments are
conducted to evaluate the proposed scheme in terms of throughput and delay.
Experimental results prove that the proposed scalable MAC outperforms other
schemes. The authors chose airtime as a slicing metric for the virtualized
wireless network since it guarantees isolation among slices. Time is divided
into fixed frames each with index $t$ and it is assigned by the AP using the
DA regime and RA regime for the periods of time given by $T_{DA}(t)\leq
T_{max}$ and $T_{f}-T_{DA}(t)$, respectively. The detailed illustration for
the frame structure is given in Fig. 12.

Figure 12: Structure of the frame for Virtualized Wireless Networks

The work in [87] presents a new contention-based MAC protocol for WSNs
referred to as, QL-MAC. The proposed protocol employs the Q-learning algorithm
to reduce energy consumption based on the actual network load in the
neighborhood by finding an efficient wake-up strategy. It benefits from a
cross-layer interaction with the network layer to enable a better
understanding of communication patterns and thus reduces energy consumption
due to idle listening and overhearing. The goal of QL-MAC is to permit the
nodes inferring each other’s behavior to accept a good active/sleep policy
scheduling. Q-learning is the most popular and powerful algorithm under the
umbrella of RL. Each node must decide whether it should be in sleep or in
active mode during each individual time slot. Each node stores a set of
Q-values associated with a particular slot inside the frame. The Q-value
provides an indication of the benefits derived by a node from being awake
throughout the associated time slot. The updates of the Q-value through time
are based on a number of specified events happening through every frame at the
same slot. Simulations are conducted using OMNET++ to compare the performance
of the proposed protocol against related MAC protocols in terms of nodes’
energy average energy consumption and the Packet Delivery Ratio (PDR). Results
show that QL-MAC outperforms other MAC protocols as it allows nodes to save
considerable energy due to the sleep/wake-up radio schedule. Also, the results
show that when the number of slots declines, generally QL-MAC exhibits
improved performance relative to the PDR, but the energy spent by the node
tends to increase.

###  V-D ALOHA-based

The work in [85] presents a Q-Learning-based MAC with informed receiving for
WSNs named ALOHA-QIR. ALOHA-QIR has the advantage of low computation,
overheads, and simplicity of ALOHA-based scheme. In addition, Q-Learning, as
an RL algorithm, is selected to avoid collisions and retransmissions facing
ALOHA-based schemes. Q-Learning stateless is employed in ALOHA-QIR to gain
experience in learning. The Q-value that exists in every node is individual
for each slot which describes the desirability of the slot selection. At the
learning time, the nodes keep jumping to distinct slots to find the perfect
one, and they want to listen to keep away from losing any information from the
earlier jumps. Informed receiving and ping packets are applied to switch nodes
to a rest mode if it could happen when it is important to make the energy
efficiency better, exceptionally when traffic operation is low. Experiments
are conducted to compare the performance of ALOHA-QIR with basic Slotted-
ALOHA. Five performance metrics are considered, normalized throughput, end-to-
end delay, the energy cost per bit throughput (indicates the average cost of
bringing data), energy cost per second (determines the proportion of consumed
energy by receiving /transmitting), and the network lifetime (compares the
cost of energy by overheads and data). Simulation results confirm that the
ALOHA-QIR achieves over double the throughput of basic Slotted-ALOHA.
Moreover, QIR-based networks can survive at least 25 times longer than
ALOHA–based networks.

Figure 13: Repeated frames and Q-values examples.

Wireless sensor networks (WSNs) exhibit a fast-growing technology with a
diverse range of applications for the military, industry, and environment
monitoring. The MAC protocol has an important part in maximizing the energy
efficiency and throughput of WSNs. Many MAC protocols have been proposed for
WSNs that significantly improve energy efficiency and throughput performance.
However, these protocols incur higher overheads and exhibit ever-increasing
complexity. Also, Performance evaluation through simulation is common but the
reasonableness of recently proposed schemes is questionable. Simpler protocols
are needed and can, despite what just has been said, provide efficient energy
communication, acceptable throughput, and acceptable delay. RL allows entities
to learn efficacious interaction strategies using trial and error in a dynamic
environment.

In [90], the authors proposed ALOHA-Q, a protocol that adopts Q-learning with
frame-based ALOHA as a strategy of slot selection which provides collision and
retransmission avoidance for single-hop networks. In particular, every node
has a unique Q-value for each slot in the frame that is updated by failure or
successful outcomes of transmission, see Fig. 13. In successful transmission,
a reward of +1 is returned otherwise the reward is -1. Slots with higher
Q-values are preferred, if multiple slots have the same Q-value, one or more
will be randomly selected. When nodes want to transmit, they will wake up,
also they will wake up when they have to receive the associated
acknowledgments (ACKs). In the experiments, a Markov model of the ALOHA
Q-learning is built to analyze its behavior of convergence. Convergence time
collected from the Markov model is compared to other related MAC protocols,
i.e. S-MAC and Z-MAC. The results of a single-hop network type show that the
proposed ALOHA-Q outperforms existing related protocols and provides fast
convergence to conditions of steady state, comparable throughput, and delay
with better energy efficiency. Also, ALOHA-Q is much simpler.

##  VI Wireless Nets

The emergence of new wireless communication systems results in a huge increase
in the resources of radio demand. The problem of spectrum shortage has been
raised due to insufficient spectrum allocation, ineffective frequency band
exploitation, and many neglected spectra. Recent studies promote the
development of dynamic spectrum access techniques as a promising solution for
the spectrum shortage problem. Several works proposed Q-learning-based Model-
dependent solutions to solve the spectrum access in dynamic environments.
However, these solutions cannot be effectively adopted in general for handling
more complex real-environment. Recently, DRL has attracted much attention
owing to its powerful learning and computation ability. DRL combines
Q-learning and neural networks to overcome Q-learning limitations for
processing large-scale models.

The work in [97] investigates the use of dynamic spectrum access to make the
most of multi-channel wireless networks. It assumes the contribution of $N$
users on $K$ channels, with each user having a choice of channel for
transmission. Every user chooses a channel and sends a packet with a specific
chance of succeeding. Every user who has attempted to transmit a packet, after
the regular time slot gets a binary observation indicating the outcome of
their sending. The optimal strategy for spectrum access is, in general,
computationally expensive. An algorithm for dynamic spectrum access based on
partially observable DRL is developed. Experimental results show that the
proposed algorithm can improve channel usage by 90%, without online
coordination, message exchange, or carrier sensing.

Improving resource utilization across networks is important because of the
unexpected increase in services and traffic. Wireless network virtualization
is one of the most promising solutions as it enables physical resource sharing
between different Service Providers (SPs). To achieve higher utilization and
lower implementation costs, virtualizing resources into different slices, each
for an SP, should be efficiently reached. To support the isolation between
SPs, the Quality of Service (QoS) constraint for every slice should be
satisfied under all conditions. One way is exact isolation, where different
slices can be kept totally isolated by adopting a Time Division Multiple
Access (TDMA) scheme. However, this allocation could result in resource
underutilization because the slice’s reserved timeshare could be partially
utilized if there are no active users in each slice. Another way is to use
random access protocols such as CSMA/CA to progressively manage the time
sharing of every slice depending upon the number of active users. That’s why
they can defeat the usage problem. On the other hand, they suffer from
isolation problems due to inevitable collisions coupling from different slice
flows. These major problems with TDMA and CSMA protocols based encourage the
development of a new multiple-access protocol to ensure the required QoS
conditions for every slice plus the resource utilization. The work in [92]
introduces a hybrid adaptive protocol of TDMA-CSMA MAC to ensure network
isolation between Service Providers (SPs) in a virtualized network. The
authors in [92] schedule the AP of high-traffic users from many slices using
TDMA while low-traffic users contend for transmission in the CSMA phase. That
is why the proposed approach can benefit from the opportunistic nature of CSMA
and the isolation advantage of TDMA. The main aim of this approach is to
provide isolation between the slices over every superframe. Specifically, it
tries to decrease the effects of actions in one slice on other slices. If the
traffic arrival statistics are not available, this scheduling will be built as
a Multi-Armed Bandit (MAB) model, in which every arm belongs to possible
scheduling. Because of arms dependency, modern policies are not directly
applicable to this problem. That’s why an index-based policy is presented
where decisions and updates are depending on learning indices given to every
user instead of every arm. In order for indices to be updated, observations
from the CSMA phase are integrated with TDMA information to add a new phase of
exploration for the MAB problem. See Fig. 14. Simulations are conducted to
evaluate the proposed protocol. The results confirm the efficiency of the
isolation policy between slices. In addition, the proposed protocol increases
the system throughput.

Figure 14: A description for the difference between Multi-Armed Bandits (MAB),
Contextual Multi-Armed Bandits (CMAB), and RL.

Contention-based protocols are the most common MAC layer access plans in
today’s wireless networks. In low-load conditions, it has been proven that
contention-based protocols perform better than resource-allocation techniques
such as TDMA and FDMA. However, with higher traffic loads, more collisions
tend to be created by random access techniques, and as a result, an urgent
need arose for scheduling protocols. Distributed Coordination Function (DCF)
protocol to reach multiple access communication for decentralized wireless
networks uses CSMA/CA. It uses a mechanism that exponentially backs off for
decreasing MAC layer collisions in the IEEE 802.11 standards. Several works in
the literature have shown that this exponential back-off mechanism and its
enhanced variants are not efficient under unknown changes such as packet
arrivals and user exit/entry. The work in [91] studies the problem of back-off
window optimization from an RL perspective and proposes algorithms of online
learning that know the back-off optimal plans under unknown changes. In the
proposed protocol, the RTS-CTS handshake mechanism is represented via an MDP
model. The proposed protocol chooses the back-off window depending upon the
optimal state of the system instead of the exponential mechanism which is
introduced in the IEEE 802.11 standards. A learning algorithm such as Post
Decision State (PDS) is proposed to increase the speed of the learning process
as it differentiates the unknown and the known components, both probabilities
of the transition of state and costs/rewards, to reach a better and faster
learning performance. If compared to conventional Q-learning algorithms, the
introduced PDS-based learning algorithm adventurous system partial information
so that less information needs to be learned. Also, it eliminates the need for
exploring actions that normally delay the learning process. In [91], the back-
off window optimization problem is addressed in a single-user setting.
Experimental results show that for a single user, a smaller back-off window is
preferred when the network load is smaller and vice versa.

A new pattern called Wireless Mesh Networks (WMNs) is developed to supply
extensive coverage of the network without infrastructure usage. In these
networks, nodes are used simply as relays to spread data from the source to
the destination using multi-hop paths. WMNs usually employ IEEE 802.11
standards whereas the MAC protocol depends on CSMA/CA and DCF. The physical
layer uses many coding and modulation techniques to support various rates.
Employing a higher transmission rate needs higher transmission power to reach
the required SINR on the receiver. This result in higher interference between
the communicating nodes and generally reduce network throughput. Several
studies are proposed in the literature to adapt the transmission rate of WMN.
However, these studies do not differentiate between packet collisions and
channel errors in case of transmission failure. The work in [89] presents a
new RL to adapt the rate of transmission for the dynamic conditions of WMNs
referred to as (RARE). The proposed algorithm uses the access probability of
the wireless channel to determine if the transmission rate will be updated or
not. It alleviates the bad impact of unnecessarily updating the transmission
rate when the transmission failure happens due to channel errors instead of
interference. The proposed, Q-learning-based, RARE is an agent-based algorithm
where each node computes the access probability of the medium-based
communication on the number of unsuccessful transmissions and the current
transmission rate. Also, every node periodically collects a Hello message from
its neighbors containing a traffic-load estimation, the channel access
probability, and the transmission rate. RL is used at every node to learn from
earlier actions whether it is important to update the transmission rate. This,
in turn, decreases the negative consequences of acquainting the transmission
rate when the throughput degradation happens by channel errors not by
interference. Experiments are conducted to compare the performance of the RARE
algorithm against other benchmark schemes. In experiments, Fisher’s Least
Significant Difference (LSD) is used as a performance metric to compare the
throughput obtained by various methods. The simulation results show that RARE
achieves higher throughput compared to the state-of-the-art algorithms for
different transmission loads and the number of contending nodes.

Figure 15: A general view of the four levels of cognition cycle: first,
information gathering, second, actions planning, third, actions, fourth,
feedback collection.

The proliferation of wireless communication technologies has resulted in an
increase in heterogeneous, co-located wireless networks with different
requirements, mobility capabilities, data rates, and coverage. Run-time
cooperation between devices from heterogeneous wireless networks becomes a
challenge. One way to support connectivity between co-located devices is to
manually group them into different sub-nets, according to their communication
technology. This approach is quite complex and inefficient. Direct cooperation
between the independent networks through shared resources such as intermediary
nodes for routing can address these problems and improve network performance.
However, the configuration problem is further complicated, as management deals
with many heterogeneous networks, distinguished by differing requirements and
capabilities. Using a cognitive entity to initiate and supervise the entire
cooperation process is the most promising solution for the above-mentioned
problems. An RL technique, Least Square Policy Iteration (LSPI), had been
proposed in [88], for high-level network optimization in heterogeneous co-
located networks. LSPI is a form of ML that collects knowledge by using a
series of trials and errors, that represent the network characteristics, to
evaluate the impact that different service combinations have on the
requirements of each network. Different decision-making policies are not
required by the proposed cognitive engine since it constantly selects
environment samples and updates the sampling matrices, see Fig. 15. Therefore,
each time a new sample is collected, LSTDQ is reinitiated using the same
policy. Compared to other work, the proposed approach does not require prior
knowledge of service impacts on network performance. Experiments are conducted
to evaluate the performance of the proposed cognitive engine against a linear
programming-based reasoning engine. Unlike other works, the proposed LSPI is
capable of adapting to dynamic network conditions and learning optimal network
configurations.

For next-generation wireless networks, efficient video streaming will play a
key role in various important applications. Unfortunately, current mobile
Internet architectures are unable to achieve Quality of Experience (QoE)
constraints for the acquired video, e.g. small zapping time, playout quality,
or limited delay of delivery. This is attributed to the limited radio
resources, the dynamic channel characteristics, and the dynamic
characteristics of video content. To address the aforementioned problems,
several works have been proposed. In the majority of these works, the control
process uses the receiver buffer and channel states. With this being said,
this information can arrive at the controller with a certain delay in real
networks. One method to address the issue of time-varying properties of both
the channel and the encoded videos is to know and update the policy of optimal
layer filtering online. RL methods are well appropriate to regularly update
the policy and the function of the state value. The authors in [86] introduce
a cross-layer control mechanism and scalable video stream to mobile receivers.
The goal of the proposed approach is to maximize the quality of the received
video while taking into account the variations in the characteristics of both
the transmitted content and the channel. To deal with the dynamic
characteristics of multimedia content and wireless channels, RL methods are
used to dynamically update optimal policy. The proposed mechanism focuses on
different classes of online RL algorithms, such as Temporal Difference (TD),
which targets to directly estimate the function of action value, i.e.
Q-function. Several experiments have been conducted to evaluate the proposed
layer filtering using both standard and real video sequences. In experiments,
the absence and delay of channel state information are considered. PSNR is
considered as the quality metric for evaluation. Results show that with the
absence or delayed channel state information, the performance of the proposed
solution is slightly degraded. Since PSNR is not enough to reflect the
perceived video quality, subjective quality metrics should be considered and
estimated at the transmitter.

Figure 16: Subnet 1 and subnet 2 with operation of MF-WT

From a control point of view, we can divide the MAC protocols into two
categories: distributed and centralized (i.e. infrastructure-based) MAC
protocols. Distributed MAC protocols achieve lower throughput compared with
centralized protocols, but multi-hop transmission is not provided by
centralized MAC protocols. This could be attributed to the hidden terminal
problem arising from the discrepancy between subnets. Providing multiple
channels or frequencies for transmission is an efficient method for
centralized MAC protocols to solve the hidden terminal problem between
subnets. A small number of investigations on systems of multi-channel concern
the architecture of multi-channel for transmission of multi-hop in centralized
wireless networks. These works rely on registering a new Wireless Terminal
(WT) and employing a large number of channels. In this way, the implementation
cost of these protocols, and the consumption power of every WT are high.
Multiple Frequency Forwarder Wireless Terminals (MF-WTs) is proposed in [81]
to enable multi-hop transmissions for centralized MAC protocols. An MF-WT is
designed to join two or more overlaying asynchronous subnets by changing these
subnets’ frequency, see Fig. 16. Despite that, many topologies of the network
have no MF-WT available, because the MF-WT must exist in the same area with
overlaying subnets. The authors in [81] suggest a mechanism of multi-hop
referred to as Adaptive Channel Switching (ACS) for centralized MAC protocols.
ACS mechanism provides efficient bandwidth utilization by preventing channel
divisions between centralized protocols subnets. It allows subnet transmission
using multi-hop and overcomes the problem of the hidden terminal by dividing
the whole bandwidth into three channels:

  * •

Relay Channel (R-channel)

  * •

Data Channel (D-channel)

  * •

Control Channel (C-channel)

The R-channel is used to relay packets to adjacent subnets by subnet boundary
stations, while D-channel is used for data packet transmissions, and C-channel
is for control-signals exchange. The proposed approach provides two distinct
operation modes, 1) Free Mode (F-mode) and 2) Restricted Mode (R-Mode). In
F-mode, stations are allowed to use the C-channel to send control signals and
use the D-channel to send data packets, they also use the R-channel to
communicate with other stations in R-mode. Stations in R-mode are restrained.
It uses just the R-channel to send data packets based on the CSMA/CA and
RTS/CTS mechanisms.

In literature, a great attention had been given to the problem of
collaboration in wireless networks. As effective collaboration among nodes
undoubtedly leads to a great improvement in the performance of the wireless
network. Different studies have been presented and shown that the problem of
collaboration can be represented as a Markov Decision Process (MDP) framework.
In these studies, each node has a local MDP comprising its own reward
functions, state, and action. The wireless nodes then collaborate to find a
near-optimal solution by dealing with the limited information with their
neighbors. The studies considered the use of distributed MDP plans for the
problem of collaboration. Although, they do not look into the methods of
distributed learning. The research in [83] presents an MDP architecture for
adapting the transmission power and probabilities. This adaptation happens in
the relay and source nodes in order to fulfill the network’s highest
throughput per unit of energy consumption. The research, also, introduces a
new learning method named the Distributed Reward and Value function (DRV). It
is a conjunction of Global Reward-based Learning (GRL) and Distributed Value
Function (DVF) methods. In the proposed (DRV) method, to maintain a balance
between the long and short-system-term rewards, both value functions and
rewards are shared among neighbors. In particular, the quick reward stresses
the present system state, and the value function is a strategic average of the
rewards. Thus, by delivering both reward and value functions at the nodes, a
complete view of the system is obtained. Simulations are conducted to evaluate
the proposed learning methods against other learning methods and
noncooperative schemes. Simulation results show that all the learning
approaches greatly surpass the not cooperative design by providing a minimum
improvement of 50%. Particularly, learning methods can exploit the dynamics of
channels, and accordingly adapt their probabilities of collaboration, to
accomplish a higher throughput per unit of consumed energy and lower network
collision.

###  VI-A Mobile Ad-hoc Networks (MANETs)

The work in [63] extends the prior efforts by dynamically weighing the pros
and cons of CSMA and TDMA for MANET. The paper introduces a hybrid CSMA-TDMA
MAC protocol called Load-Adaptive MAC (LA-MAC). It is specifically planned and
executed on MANETs formed by MIMO-equipped USRP nodes. By dynamically
switching its operating mode, the proposed protocol is aimed to conduct
similarly to CSMA under low collision conditions and TDMA under high collision
conditions. Particularly, the LA-MAC protocol offers better performance using
a cross-layer PHY-MAC design to command the mode-switching behavior of the
protocol based on real collisions as opposed to MAC frame corruption
associated with wireless channel effects such as fading. Experiments are
conducted on a MANET testbed and two different metrics are used to investigate
the LA-MAC performance, Maximum Achievable Data Throughput (MADT) and Round
Trip Time (RTT). The results show that, in contrast to other existing
protocols, the throughput of LA-CSMA is almost independent of the number of
transmitters. In addition, every time slot can be used for transmission. The
results also show that the average latency of LA-CSMA is almost 10% lower than
other protocols.

In [96], Current wireless networks are very dynamic and support numerous
applications. As an example, a cellular network carries a variety of traffic
such as multimedia messages, text, webpage requests, video calls, and voice
calls. Each of these traffics has its own requirements and characteristics.
Furthermore, it is necessary that the network adapts its attitude based on the
application requirements. On the other hand, the requirement for adaptability
also occurs in the sublayer of MAC, because it is not possible for an
individual protocol to achieve orthogonal requirements such as power
consumption, latency, availability, bandwidth, and security. Furthermore,
devices and networks should adjust to the context of communication in order to
utilize resources usages. The work in [96] introduces a highly adaptable
protocol platform for the MAC sublayer, called FS-MAC. FS-MAC dynamically
switches the used MAC protocol in the network, to preserve using the protocol
that is more effective for the current network status. The switch protocol
depends on a set of fuzzy rules, that can be adapted according to the network
administrators and application demands. The proposed architecture is
extensible for additional MAC protocols. FS-MAC is not considered to be a new
protocol for MAC, rather it authorizes the use of many protocols in the MAC
layer. Each of these protocols will be used in the appropriate conditions
where it is highest effective. Extensibility is the most intrinsic feature of
the proposed platform as a new monitoring metric, that enriches the decision,
and new MAC protocols could be added in any later phases. Furthermore, the
decision rules could be changed by the network administrator to deal with new
applications or to meet the prerequisite of the network. The rules are
centralized for the best protocol. In decentralized networks, i.e. Mesh or ad-
hoc networks, a leader election algorithm to choose the coordinator can be
employed. The proposed framework is composed of three main modules: sensing,
decision, and change modules. The sensing module assembles data to determine
which MAC protocol must be active. The Decision module computes, using fuzzy
logic, the adjustability of every protocol to the current network after
getting the information from the Sensing module. Simplicity and extensibility
are the key features of fuzzy logic over other intelligent approaches. In
addition, Fuzzy regulation captures the domain knowledge from domain experts,
so the system does not require pre-training, and new rules can be added
without touching the existing rules. The change module inspects the need to
switch the currently used protocol and allows the system to change the MAC
protocol only if the adjustability of the current protocol is less than the
adjustability of the finest protocol plus a threshold. A testbed has been used
to measure the platform’s capability to respond to different congestion
levels. The load in every station is held constant during the experiments
while a variable number of transmitting stations is considered. Results show
that FS-MAC has delay values and throughput like those of the finest fixed
protocol for every scenario, with only 2% overhead.

The work in [82] proposes a new Scalable Cognitive Routing Protocol (SCRP) to
save routing overhead for mobile ad-hoc networks. The suggested SCRP exploits
a flooding protocol with intelligence. A neural network-based approach is
approved to make nodes familiar with history. Every node forecast future link
status based on previous experience and flood RREQ packets along with the
predicted strong links and over-predicted good frequencies.

###  VI-B Machine to Machine (M2M)

M2M networks are expected to be broadly used in many widespread IoT
applications such as power grid systems, transportation systems, and health
care. A key characteristic of M2M networks is the massive number of devices
and parallel network access attempts from these devices. Contention-based MAC
protocols are not convenient to provide scalable, flexible, and structured
automatic communication for a dense heterogeneous M2M network. Combining the
advantages of contention-based and reservation-based MAC protocols in a hybrid
scheme has received considerable interest. An efficient and scalable MAC
protocol is essential for M2M networks to enable a huge number of devices to
concurrently access the channel. Current wireless M2M networks experience more
diverse traffic characteristics than usual networks. To enhance the channel
usage for M2M networks, the traffic statistical information might be leveraged
to efficiently select a MAC protocol or to configure a certain protocol in
response to differing conditions. The traffic statistical information, or
parameters, may not be readily available or may need to be gathered over time.
Therefore, employing a learning algorithm to acquire traffic statistics is
crucial.

In [67] and [68] the authors propose a frame-based Hybrid MAC protocol for M2M
networks which combines the advantages of both contention-based and
reservation-based protocols. In the proposed approach, frames are divided into
two parts: Transmission Only Period (TOP), and the Contention Only Period
(COP). COP depends on the CSMA/CA access method while the TOP delivers TDMA
type of communications. The devices first compete for TOP transmission slots.
Transmission time slots during the TOP will be given to the winning devices.
To achieve the optimal balance in each frame between the transmission and
contention period, an optimization problem will be developed for throughput
maximization. Experiments are conducted to evaluate the proposed Hybrid MAC
protocol with the contention-based protocol (i.e., slotted ALOHA) and the
reservation-based protocol (i.e., TDMA). The comparison considered various
metrics including throughput, service delay, and average transmission. The
obtained results indicate that the proposed hybrid protocol successfully
reduces collision probabilities and increases the overall device’s throughput.
On the other hand, Slotted ALOHA has a good performance only in low-loaded
conditions, and TDMA has a good performance only in high-loaded conditions.
The hybrid protocol achieves a lower delay than Slotted ALOHA and on-the-bar
with TDMA.

###  VI-C Cognitive Radio (CR)

Cognitive radio (CR) has been raised as a promising solution for the problem
of spectrum scarcity in wireless communications. CR improves the efficiency of
spectrum usage by allowing CR users to access the gaps in the spectrum.
However, higher CR intelligence techniques are still needed, particularly for
a number of individual cases of radio environment, to reach the goal of smart
radios along with the goal of CR transmissions [100]. Wireless technologies
are becoming more important and involved in various aspects of our daily life.
The increasing number of wireless applications, from satellite control to home
appliances, creates problems such as spectrum shortages and a lack of radio
resources for those who are most in need. In a Cognitive Radio ad-hoc Network
(CRN), the SUs interfere with each other because of the difficulty of
synchronization and thus the protocol has to address this issue. In this
subsection, we briefly introduce some of the AI-based works in this direction.

The work in [93] suggests a Multi-Channel Cognitive MAC Protocol for CR
wireless ad-hoc networks. The proposed protocol uses an RL model for secondary
user (SU) channel selection based on primary user (PU) traffic and with small
computational requirements to keep SUs synchronized and avoid collisions that
may happen with the PUs. In particular, the authors consider network
architecture that has clusters of SU nodes experiencing the same PUs
existence. Inside each cluster, local traffic can be exchanged, while inter-
cluster communication is achieved by the gateway node. The proposed MAC
protocol uses a common per-cluster resource reservation control channel to
restrict the collected interference of SUs and to deal with the problems of
hidden terminals. The suggested protocol depends on the synchronization
reached inside the cluster by using the distributed scheme since the sender
and receiver are aware of the existence of the identical PUs and they share
identical probabilities of channel selection. Transmissions are identified and
SUs are synchronized because PUs and Sus have no collisions between them.
Energy acquisition and cyclostationary feature acquisition can be used for
spectrum acquisition. Simulations are conducted using the OMNET simulator to
evaluate the performance of the proposed protocol against the CREAM-MAC, a
protocol introduced in [101]. For various PU burst lengths, the collected
throughput is analyzed. When the bursts are generated every 2s and 0.1s, the
proposed protocol is less influenced by the changes in PU burst length as it
applies the maximum of the spectrum holes for each PU burst generation rate
that learns, and differs from CREAM-MAC as its performance falls substantially
by the PU burst length. Simulation results also show that the proposed
protocol forecasts and completely evades PUs collisions.

Based on the way of advertising the FCL between the participating cognitive
nodes, Cognitive Radio MAC protocols can be classified as a Global CCC (GCCC)
and non-GCCC. The authors in [64] present a novel Dynamic De-Centralized
Hybrid MAC protocol, referred to as “DDH-MAC”, for Cognitive Radio Networks.
DDH-MAC protocol lies between GCCC and non-GCCC categories of cognitive radio
MAC protocols. It supports many security levels and two selection levels. The
first level of selection lets cognitive nodes learn about deciding which two
white spaces to apply BCCH and PCCH. At the moment when the BCCH and PCCH are
decided, switches cognitive nodes exchange control information through the
empty space, and the selected second level starts. Four levels of security are
provided by DDH-MAC, by enciphering the BF using either private or public
cryptology designs, the security first level has been achieved and the message
integrity is gained using codes of message authentication. Secret exchange of
the FCL in PCCH, which is accessible locally and is only acknowledged to the
engaging cognitive nodes, presents a second level of security. An additional
security level is accomplished by the addition of a timestamp plus its hash
value in the precise data frames transmitted by the CR nodes. Finally, DDH-MAC
dynamically adapts new PCCH for every transaction. For evaluation, the time of
pre-transmission for scenarios of different DDH-MAC has been calculated and
compared with other existing CR MAC protocols. The results show that the time
of pre-transmission is 20% superior on average for DDH-MAC and this also will
help DDH-MAC to achieve improved QoS as the nodes do not need to wait. In CR
networks, CR users have to incrementally rearrange the properties of their MAC
layer in response to the changes of the outside radio settings to reach an
improved network performance. Different computational intelligence is required
to help CR nodes to decide which MAC protocol should be used.

In [37], the authors consider the MAC protocols as radio specifications and
present a modern technique named MAC protocol identification to execute the
cognitive cycle for CR. The proposed approach acts in the execution of the
reconstruct/adaptive MAC. It enables CR users to verify and ascertain the MAC
protocol type and the MAC layer specifications of any network users, involving
CR and PUs. Identification results enable the CR user to enhance the spectrum
holes fulfillment capability, decrease the interference to reactivate the
original users, and promote communications among diverse CR networks. Also,
the proposed approach helps CR users to enhance the spectrum sensing
frequency/period to decrease power usage. Two experiments are conducted to
evaluate the performance of packet length prediction and protocol
identification using a support vector machine model. The experiment results
show that the proposed approach achieves 92% accuracy in packet length
prediction. While the MAC protocol identification accuracy reaches 90% with
increased PU traffic load.

Figure 17: Spectrum holes’ allocation of CR transmissions.

The authors in [36] investigate recognizing the MAC protocol for applications
in cognitive networks considering TDMA, CSMA/CA, and both pure and slotted
ALOHA networks. MAC recognition allows CR users to recognize and sense the MAC
protocol types for available transmission. Consequently, the identification
results enable CR users to adapt their transmission specifications to enhance
spectrum exploitation and decrease interference between secondary and primary
users. In particular, the received signal strength (RSS) is considered the
main feature of CR users. It is used to detect spectrum availability through
an SVM classifier. Which in turn increases the detection probability and
reduces the false alarms probability. When a CR user attempts to use a
spectrum hole, time duration and frequency range are treated as channel access
specifications. The frequency range of a spectrum hole is usually determined
by spectrum acquisition; however, the time range information is not used.
Unlike spectrum capture, MAC protocol identification allows CR users to match
their transmission to the timing pattern of a spectrum hole, see Fig. 17. In
literature, research on CR networks suggests that the users in primary
networks are non-reactive which is why the primary transmission processes are
not influenced by the transmission of CR users. However, in a reactive primary
network, the appearance and transmission of CR users will seriously interfere
with PU. To address the interference with the responsive PUs, a MAC protocol
recognition is required to detect the protocol type of a primary network, and
the next coexisting access scheme of CR is applied. Experiments are conducted
to evaluate the MAC identification performance considering TDMA, CSMA/CA, pure
ALOHA, and slotted ALOHA MAC protocols. Power and time components are obtained
and used by SVMs to detect the type of MAC protocol. Each CR user keeps
sensing the channel and recording the RSS and channel state durations. Power
mean and variance are, in this case, used as power components. The minimum,
median, and maximum values of the recorded channel idle and channel busy
durations, are used as time features. Experiments show that using the same
kernel function the used SVM has different performance for the four MAC
protocols. S-ALOHA is easier to identify than the other three protocols as the
high contention behavior provides more accurate power features. Experiments
also considered the use of different kernel functions for the same MAC
protocol. The results show that the linear and polynomial kernels achieve more
effectiveness than the radial base kernel. That implies that kernel selection
is critical since it influences the identification accuracy.

##  VII Body Sensor Networks (BSNs)

Body Sensor Networks (BSNs) are composed of many sensor nodes placed on/around
a human body to send and/or receive data through wireless connections. A
commonly used low-power BSNs wireless protocol is IEEE 802.15.4 (ZigBee) which
supports healthcare applications [102]. Most of these applications use CSMA/CA
as the standard MAC scheduling scheme for IEEE 802.15.4. However, CSMA/CA
provides unacceptable performance in dense environments with multiple co-
located BSNs. Time-slotted channel hopping (TSCH) is a well-suited MAC
protocol for healthcare applications as it incorporates access to time-slotted
and hopping of multi-channel capabilities to deliver applications with
deterministic latency. Also, it improves the reliability of wireless
communications in the presence of heterogeneous technologies. In this
subsection, we present the recent AI-based MAC work targeting BSN [22].

In [72] a joint sampling plan of dynamic sensor plus a BSNs dynamic MAC
scheduling plan based on TSCH has been presented for a healthcare monitoring
system. The adaptive MAC scheduler depends on a state machine model that
responds to the sensors-generated dynamic healthcare traffic. Additional time
slots for the emergency sensors had been allocated automatically by the MAC
scheduler. This emergency allocation allows reliable transmissions for high-
resolution sensor data. These extra time slots are released back, when the
emergency period is expired, to the pool of available time slots of the whole
network. Such a mechanism solves the problem of allocating available time
slots of TSCH to the required sensors while maintaining the other sensors’
QoS. Experiments are conducted to evaluate the adaptive MAC scheduler against
other MAC schedulers for TSCH in terms of Packet Delivery Ratio (PDR) and
Goodput (G). The results show that the proposed MAC scheduler maintains a
great communication channel QoS, nearly 100% PDR constantly, and obtains a
great Goodput ratio in contrast to the other existing MAC schedulers.

##  VIII Spectrum Sensing

Dynamic Spectrum Access (DSA) was introduced to deal with the largely
increased need for resources of the finite spectrum. DSA is proposed to open
spectrum chunks that are currently allocated for licensed PUs to other
unlicensed CR SUs. Existing spectrum sensing approaches for CR DSA are
generally categorized into two categories: energy-based detection approaches
and signal classification approaches. Apart from the instantaneous spectral
and temporal occupancy of the PU, energy-based approaches did not provide much
information. This is considered as an advantage to drive the sensing policy,
nevertheless, it does not adequately express PU health. Higher-level
information about the PU is provided by signal classification approaches, but,
in some cases, requires complex and computationally intensive features and
supervised training. Following we present some works that explore spectrum
sensing for MAC techniques.

In [73], the authors present a learning approach for spectrum acquisition,
where the PU application protocol is discovered such that PU knowledge of the
application protocol is identified using easy functions for energy detector.
The presented approach allows an SU to automatically identify access
opportunities only using externally observable energy detector features. This
approach includes an unsupervised nonparametric Bayesian Hierarchical
Dirichlet Process Hidden Markov Model (HDP-HMM) training method along with
efficient HMM recursions for log classification, log state detection, and
anomaly detection. Experimental results, on a wireless network testbed, show
that hidden states could be learned by HDP-HMM. Hidden states correspond to
the states of the major user application layer protocol. HDP-HMM looks for PU
traffic abnormality occurring by SU interference. There have been prior
attempts for exploiting Deep Reinforcement Learning (DRL) to solve the MAC
problems. Compared to RL-based MAC, DRL converges more quickly and is highly
robust. To approximate the state/action/value-function in DRL, a three-layer,
ReLU-activated, neural network has been used.

Figure 18: A mix of many nodes and DRL nodes in a multiple access
heterogeneous system.

In [74], the authors proposed a DRL-based MAC protocol known as DLMA, see Fig.
18. DLMA considers time slot systems and the problem of dividing time slots
between heterogeneous wireless networks. A key characteristic of DLMA is while
operating in a heterogeneous environment, it can learn how to reach an overall
goal through an array of state/action/reward monitoring. Particularly, without
knowing the detailed operating mechanisms of coexisting MACs, DLMA achieves
near-optimal performance in relation to the goal. Experiments have been
performed to evaluate the DLMA approach against existing RL-based approaches.
The results show that DRL converges to the perfect throughput largely greatly
faster than RL. Results also show that the DRL node can learn the proper
strategy to reach the best throughputs regardless of the MAC protocol type and
the number of other nodes.

##  IX Security

In cybersecurity, Intrusion Detection Systems (IDS) are used for predicting
and detecting threats before they lead to larger security incidents. Two main
classes of IDS are widely used, namely Signature-based detection and anomaly-
based detection. In signature-based detection, a database of various known
attack signatures will be maintained, while anomaly-based detection examines
and detects intruder behavior by reviewing past activity and checking for
deviations from normal traffic behavior. IDS uses various ML approaches
specifically, classification techniques to detect cyberattacks.

The authors in [75] consider multi-class classification for the Aegean Wi-Fi
Intrusion Dataset (AWID) where classes represent $17$ types of the IEEE 802.11
MAC Layer attacks. Four feature groups are considered $32$, $10$, $6$, and $5$
based on various feature selection and reduction algorithms. Firstly, based on
reliable and accurate manual feature selection and recommendations from prior
works, a total of 32 attributes were selected. The Correlation Feature
Selection (CFS) with the Best First Search (BFS) method of forward direction
is used to evaluate the feature importance and select the $10$ highest
correlated attributes. For the second feature group, Harmony Search (HS)
algorithm with the cost-sensitive subset evaluator is also employed to
evaluate the features and only seven attributes are selected. Finally, the CFS
algorithm along with the Harmony Search technique is adopted for the feature
selection and dimensionality reduction process, and five attributes are
selected. Experiments are conducted to evaluate the effect of the feature
selection/reduction on the performance of the classifiers. Seven well-known
classification algorithms (i.e., J48, OneR, Naive Bayes, Random Forest, Simple
Logistic, Bagging, and Multi-Layer Perceptron) had been evaluated through the
selected feature group sets. The results show that, in terms of accuracy and
processing time, optimal attribute selection/reduction leads to better
results, which is critical for real-time applications.

##  X Conclusion

Over the past decade, there has been a marked advancement in the field of
Machine Learning (ML) and Deep Learning (DL) techniques. These developments
have had a significant impact on a wide range of industries. In the realm of
wireless communications, the utilization of ML techniques has been implemented
to enhance various Medium Access Control (MAC) protocols. This is evidenced by
the growing number of publications that utilize the proven capabilities of ML
to extend the existing MAC techniques. The present study conducts a survey of
ML-based MAC techniques from 2012 to 2022. Additionally, this survey provides
an in-depth tutorial on ML techniques, various MAC protocols, and design
issues related to MAC protocols. The survey also presents a multi-dimensional
taxonomy for the surveyed work, which is based on three main criteria: 1) the
machine learning technique adopted in the work, 2) the application area, and
3) the objective of the MAC protocol.

## References

  * [1] M. Hassaballah and A. I. Awad, _Deep learning in computer vision: principles and applications_. CRC Press, 2020.
  * [2] L. Jiao and J. Zhao, “A survey on the new generation of deep learning in image processing,” _IEEE Access_ , vol. 7, pp. 172 231–172 263, 2019.
  * [3] D. W. Otter, J. R. Medina, and J. K. Kalita, “A survey of the usages of deep learning for natural language processing,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 32, no. 2, pp. 604–624, 2020.
  * [4] Y. LeCun and et al., “Deep learning,” _Nature_ , vol. 521, no. 7553, pp. 436–444, 2015.
  * [5] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” _Communications of the ACM_ , vol. 60, no. 6, pp. 84–90, 2017.
  * [6] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot _et al._ , “Mastering the game of go with deep neural networks and tree search,” _Nature_ , vol. 529, no. 7587, pp. 484–489, 2016.
  * [7] P. Luong, F. Gagnon, L.-N. Tran, and F. Labeau, “Deep reinforcement learning-based resource allocation in cooperative uav-assisted wireless networks,” _IEEE Transactions on Wireless Communications_ , vol. 20, no. 11, pp. 7610–7625, 2021.
  * [8] Z. Wang, M. Eisen, and A. Ribeiro, “Learning decentralized wireless resource allocations with graph neural networks,” _arXiv preprint arXiv:2107.01489_ , 2021.
  * [9] E. Bobrov, D. Kropotov, H. Lu, and D. Zaev, “Massive mimo adaptive modulation and coding using online deep learning algorithm,” _IEEE Communications Letters_ , 2021.
  * [10] M. Hussien, M. F. Ahmed, G. Dahman, K. K. Nguyen, M. Cheriet, and G. Poitau, “Towards more reliable deep learning-based link adaptation for wifi 6,” in _IEEE International Conference on Communications (ICC)_. IEEE, 2021, pp. 1–6.
  * [11] H. Huang, Y. Peng, J. Yang, W. Xia, and G. Gui, “Fast beamforming design via deep learning,” _IEEE Transactions on Vehicular Technology_ , vol. 69, no. 1, pp. 1065–1069, 2019.
  * [12] H. Hojatian, J. Nadal, J.-F. Frigon, and F. Leduc-Primeau, “Unsupervised deep learning for massive mimo hybrid beamforming,” _IEEE Transactions on Wireless Communications_ , vol. 20, no. 11, pp. 7086–7099, 2021.
  * [13] F. Meng, P. Chen, L. Wu, and X. Wang, “Automatic modulation classification: A deep learning enabled approach,” _IEEE Transactions on Vehicular Technology_ , vol. 67, no. 11, pp. 10 760–10 772, 2018.
  * [14] Y. Wang, J. Yang, M. Liu, and G. Gui, “Lightamc: Lightweight automatic modulation classification via deep learning and compressive sensing,” _IEEE Transactions on Vehicular Technology_ , vol. 69, no. 3, pp. 3491–3495, 2020.
  * [15] M. Soltani, V. Pourahmadi, A. Mirzaei, and H. Sheikhzadeh, “Deep learning-based channel estimation,” _IEEE Communications Letters_ , vol. 23, no. 4, pp. 652–655, 2019.
  * [16] Q. Hu, F. Gao, H. Zhang, S. Jin, and G. Y. Li, “Deep learning for channel estimation: Interpretation, performance, and comparison,” _IEEE Transactions on Wireless Communications_ , vol. 20, no. 4, pp. 2398–2412, 2020\.
  * [17] M. Hussien, K. K. Nguyen, and M. Cheriet, “Self-supervised learning for csi compression in fdd massive mimo systems,” _IEEE Communications Letters_ , 2022.
  * [18] Z. Liu, L. Zhang, and Z. Ding, “An efficient deep learning framework for low rate massive mimo csi reporting,” _IEEE Transactions on Communications_ , vol. 68, no. 8, pp. 4761–4772, 2020.
  * [19] M. Hussien, K. K. Nguyen, and M. Cheriet, “Prvnet: variational autoencoders for massive mimo csi feedback,” _arXiv preprint arXiv:2011.04178_ , 2020\.
  * [20] X. Ye, Y. Yu, and L. FU, “Deep reinforcement learning based mac protocol for underwater acoustic networks,” _IEEE Transactions on Mobile Computing_ , 2020\.
  * [21] A. Ranjha, G. Kaddoum, M. Rahim, and K. Dev, “Urllc in uav-enabled multicasting systems: A dual time and energy minimization problem using uav speed, altitude and beamwidth,” _Computer Communications_ , vol. 187, pp. 125–133, 2022.
  * [22] H. K. Narsani, A. Ranjha, K. Dev, F. H. Memon, and N. M. F. Qureshi, “Leveraging uav-assisted communications to improve secrecy for urllc in 6g systems,” _Digital Communications and Networks_ , 2022.
  * [23] D. M. Abdullah and S. Y. Ameen, “Enhanced mobile broadband (embb): A review,” _Journal of Information Technology and Informatics_ , vol. 1, no. 1, pp. 13–19, 2021.
  * [24] Z. Dawy, W. Saad, A. Ghosh, J. G. Andrews, and E. Yaacoub, “Toward massive machine type cellular communications,” _IEEE Wireless Communications_ , vol. 24, no. 1, pp. 120–128, 2016.
  * [25] P. Naik and K. M. Sivalingam, “A survey of mac protocols for sensor networks,” in _Wireless Sensor Networks_. Springer, 2004, pp. 93–107.
  * [26] A. Bachir, M. Dohler, T. Watteyne, and K. K. Leung, “Mac essentials for wireless sensor networks,” _IEEE Communications Surveys & Tutorials_, vol. 12, no. 2, pp. 222–248, 2010.
  * [27] N. Z. binti Zubir, A. F. Ramli, and H. Basarudin, “Optimization of wireless sensor networks mac protocols using machine learning; a survey,” in _International Conference on Engineering Technology and Technopreneurship (ICE2T)_. IEEE, 2017, pp. 1–5.
  * [28] P. Huang, L. Xiao, S. Soltani, M. W. Mutka, and N. Xi, “The evolution of mac protocols in wireless sensor networks: A survey,” _IEEE Communications Surveys & Tutorials_, vol. 15, no. 1, pp. 101–120, 2012.
  * [29] P. H. Isolani, M. Claeys, C. Donato, L. Z. Granville, and S. Latré, “A survey on the programmability of wireless mac protocols,” _IEEE Communications Surveys & Tutorials_, vol. 21, no. 2, pp. 1064–1092, 2018.
  * [30] V. L. Quintero, C. Estevez, M. E. Orchard, and A. Pérez, “Improvements of energy-efficient techniques in wsns: a mac-protocol approach,” _IEEE Communications Surveys & Tutorials_, vol. 21, no. 2, pp. 1188–1208, 2018.
  * [31] C. Zhang, P. Patras, and H. Haddadi, “Deep learning in mobile and wireless networking: A survey,” _IEEE Communications Surveys & Tutorials_, vol. 21, no. 3, pp. 2224–2287, 2019.
  * [32] Y. Sun, M. Peng, Y. Zhou, Y. Huang, and S. Mao, “Application of machine learning in wireless networks: Key techniques and open issues,” _IEEE Communications Surveys & Tutorials_, vol. 21, no. 4, pp. 3072–3108, 2019.
  * [33] F. Z. Djiroun and D. Djenouri, “MAC Protocols with Wake-up Radio for Wireless Sensor Networks: A Review,” _IEEE Communications Surveys & Tutorials_, vol. 19, no. 1, pp. 587–618, 2016.
  * [34] M. Hussien, K. K. Nguyen, and M. Cheriet, “Fault-tolerant 1-bit representation for distributed inference tasks in wireless iot,” in _2021 17th International Conference on Network and Service Management (CNSM)_. IEEE, 2021, pp. 427–431.
  * [35] Z. Yang, Y.-D. Yao, S. Chen, H. He, and D. Zheng, “Mac protocol classification in a cognitive radio network,” in _Annual Wireless and Optical Communications Conference (WOCC 2010)_. IEEE, 2010, pp. 1–5.
  * [36] S. Hu, Y.-D. Yao, and Z. Yang, “Mac protocol identification using support vector machines for cognitive radio networks,” _IEEE Wireless Communications_ , vol. 21, no. 1, pp. 52–60, 2014.
  * [37] ——, “Mac protocol identification approach for implement smart cognitive radio,” in _IEEE International Conference on Communications (ICC)_. IEEE, 2012, pp. 5608–5612.
  * [38] E. Fix, _Discriminatory analysis: nonparametric discrimination, consistency properties_. USAF School of Aviation Medicine, 1951.
  * [39] L. E. Peterson, “K-nearest neighbor,” _Scholarpedia_ , vol. 4, no. 2, p. 1883, 2009.
  * [40] V. Garcia, E. Debreuve, and M. Barlaud, “Fast k nearest neighbor search using gpu,” in _IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops_. IEEE, 2008, pp. 1–6.
  * [41] R. O. Duda, P. E. Hart, and D. G. Stork, _Pattern classification_. John Wiley & Sons, 2012.
  * [42] S. A. Dudani, “The distance-weighted k-nearest-neighbor rule,” _IEEE Transactions on Systems, Man, and Cybernetics_ , no. 4, pp. 325–327, 1976.
  * [43] C. M. Bishop, _Pattern recognition and machine learning_. springer, 2006.
  * [44] T. Kanungo, D. M. Mount, N. S. Netanyahu, C. D. Piatko, R. Silverman, and A. Y. Wu, “An efficient k-means clustering algorithm: Analysis and implementation,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 24, no. 7, pp. 881–892, 2002.
  * [45] M. Ester, H.-P. Kriegel, J. Sander, X. Xu _et al._ , “A density-based algorithm for discovering clusters in large spatial databases with noise.” in _KDD_ , vol. 96, no. 34, 1996, pp. 226–231.
  * [46] T. K. Moon, “The expectation-maximization algorithm,” _IEEE Signal Processing Magazine_ , vol. 13, no. 6, pp. 47–60, 1996.
  * [47] J. A. Hartigan and M. A. Wong, “Algorithm as 136: A k-means clustering algorithm,” _Journal of the Royal Statistical Society_ , vol. 28, no. 1, pp. 100–108, 1979.
  * [48] X. Li, S. Wang, and Y. Cai, “Tutorial: Complexity analysis of singular value decomposition and its variants,” _arXiv preprint arXiv:1906.12085_ , 2019\.
  * [49] A. J. Izenman, “Linear discriminant analysis,” in _Modern Multivariate Statistical Techniques_. Springer, 2013, pp. 237–280.
  * [50] S. Wojtowytsch and E. Weinan, “Can shallow neural networks beat the curse of dimensionality? a mean field training perspective,” _IEEE Transactions on Artificial Intelligence_ , vol. 1, no. 2, pp. 121–129, 2020.
  * [51] M. Ringnér, “What is principal component analysis?” _Nature Biotechnology_ , vol. 26, no. 3, pp. 303–304, 2008.
  * [52] J. Shlens, “A tutorial on principal component analysis,” _arXiv preprint arXiv:1404.1100_ , 2014.
  * [53] R. S. Sutton and A. G. Barto, _Reinforcement learning: An introduction_. MIT press, 2018.
  * [54] K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, “Deep reinforcement learning: A brief survey,” _IEEE Signal Processing Magazine_ , vol. 34, no. 6, pp. 26–38, 2017.
  * [55] S. Khisa and S. Moh, “Medium access control protocols for the internet of things based on unmanned aerial vehicles: A comparative survey,” _Sensors_ , vol. 20, no. 19, p. 5586, 2020.
  * [56] N. Abramson, “Development of the alohanet,” _IEEE Transactions on Information Theory_ , vol. 31, no. 2, pp. 119–123, 1985.
  * [57] L. G. Roberts, “Aloha packet system with and without slots and capture,” _ACM SIGCOMM Computer Communication Review_ , vol. 5, no. 2, pp. 28–42, 1975\.
  * [58] L. Kleinrock and F. Tobagi, “Packet switching in radio channels: Part i-carrier sense multiple-access modes and their throughput-delay characteristics,” _IEEE Transactions on Communications_ , vol. 23, no. 12, pp. 1400–1416, 1975.
  * [59] M. Takata, M. Bandai, and T. Watanabe, “A mac protocol with directional antennas for deafness avoidance in ad hoc networks,” in _IEEE Global Communications Conference (GLOBECOM)_. IEEE, 2007, pp. 620–625.
  * [60] A. Ranjha and G. Kaddoum, “Urllc-enabled by laser powered uav relay: A quasi-optimal design of resource allocation, trajectory planning and energy harvesting,” _IEEE Transactions on Vehicular Technology_ , vol. 71, no. 1, pp. 753–765, 2021.
  * [61] ——, “Quasi-optimization of distance and blocklength in urllc aided multi-hop uav relay links,” _IEEE Wireless Communications Letters_ , vol. 9, no. 3, pp. 306–310, 2019.
  * [62] H. Wang, X. Zhang, F. Naït-Abdesselam, and A. Khokhar, “Cross-layer optimized mac to support multihop qos routing for wireless sensor networks,” _IEEE Transactions on Vehicular Technology_ , vol. 59, no. 5, pp. 2556–2563, 2010.
  * [63] W. Hu, H. Yousefi’zadeh, and X. Li, “Load adaptive mac: a hybrid mac protocol for mimo sdr manets,” _IEEE Transactions on Wireless Communications_ , vol. 10, no. 11, pp. 3924–3933, 2011.
  * [64] M. A. Shah, G. A. Safdar, and C. Maple, “Ddh-mac: a novel dynamic de-centralized hybrid mac protocol for cognitive radio networks,” in _RoEduNet International Conference: Networking in Education and Research_. IEEE, 2011, pp. 1–6.
  * [65] M. H. S. Gilani, I. Sarrafi, and M. Abbaspour, “An adaptive csma/tdma hybrid mac for energy and throughput improvement of wireless sensor networks,” _Ad Hoc Networks_ , vol. 11, no. 4, pp. 1297–1304, 2013.
  * [66] M. Sha, R. Dor, G. Hackmann, C. Lu, T.-S. Kim, and T. Park, “Self-adapting mac layer for wireless sensor networks,” in _IEEE Real-Time Systems Symposium_. IEEE, 2013, pp. 192–201.
  * [67] L. Yi, C. Yuen, J. Chen, and X. Cao, “A scalable hybrid mac protocol for massive m2m networks,” in _IEEE Wireless Communications and Networking Conference (WCNC)_. IEEE, 2013, pp. 250–255.
  * [68] Y. Liu, C. Yuen, X. Cao, N. U. Hassan, and J. Chen, “Design of a scalable hybrid mac protocol for heterogeneous m2m networks,” _IEEE Internet of Things Journal_ , vol. 1, no. 1, pp. 99–111, 2014.
  * [69] A. N. Alvi, S. H. Bouk, S. H. Ahmed, M. A. Yaqub, N. Javaid, and D. Kim, “Enhanced tdma based mac protocol for adaptive data control in wireless sensor networks,” _Journal of Communications and Networks_ , vol. 17, no. 3, pp. 247–255, 2015.
  * [70] A. N. Alvi, S. H. Bouk, S. H. Ahmed, M. A. Yaqub, M. Sarkar, and H. Song, “Best-mac: Bitmap-assisted efficient and scalable tdma-based wsn mac protocol for smart cities,” _IEEE Access_ , vol. 4, pp. 312–322, 2016.
  * [71] M. Qiao, H. Zhao, S. Wang, and J. Wei, “Mac protocol selection based on machine learning in cognitive radio networks,” in _International Symposium on Wireless Personal Multimedia Communications (WPMC)_. IEEE, 2016, pp. 453–458.
  * [72] M. V. Ngo, Q. D. La, D. Leong, and T. Q. Quek, “User behavior driven mac scheduling for body sensor networks,” in _IEEE International Conference on e-Health Networking, Applications, and Services (Healthcom)_. IEEE, 2017, pp. 1–6.
  * [73] G. Ford, S. Mason, K. Rigney, and M. Kam, “Enhanced spectrum awareness using bayesian nonparametric pattern recognition techniques,” in _Annual Conference on Information Sciences and Systems (CISS)_. IEEE, 2017, pp. 1–6.
  * [74] Y. Yu, T. Wang, and S. C. Liew, “Deep-reinforcement learning multiple access for heterogeneous wireless networks,” _IEEE Journal on Selected Areas in Communications (JSAC)_ , vol. 37, no. 6, pp. 1277–1290, 2019.
  * [75] R. Abdulhammed, M. Faezipour, A. Abuzneid, and A. Alessa, “Enhancing wireless intrusion detection using machine learning classification with reduced attribute sets,” in _International Wireless Communications & Mobile Computing Conference (IWCMC)_. IEEE, 2018, pp. 524–529.
  * [76] A. D. Shoaei, M. Derakhshani, and T. Le-Ngoc, “Reconfigurable and traffic-aware mac design for virtualized wireless networks via reinforcement learning,” _IEEE Transactions on Communications_ , vol. 67, no. 8, pp. 5490–5505, 2019.
  * [77] B. Yang, X. Cao, Z. Han, and L. Qian, “A machine learning enabled mac framework for heterogeneous internet-of-things networks,” _IEEE Transactions on Wireless Communications_ , vol. 18, no. 7, pp. 3697–3712, 2019\.
  * [78] M. Derakhshani, Y. Khan, D. T. Nguyen, S. Parsaeefard, A. D. Shoaei, and T. Le-Ngoc, “Self-organizing tdma: a distributed contention-resolution mac protocol,” _IEEE Access_ , vol. 7, pp. 144 845–144 860, 2019.
  * [79] A. Chilmulwar and V. Sinha, “A novel machine learning based mac scheduler algorithm using arima,” in _IEEE Annual Consumer Communications & Networking Conference (CCNC)_. IEEE, 2019, pp. 1–8.
  * [80] G. Franchino, G. Buttazzo, and M. Marinoni, “An energy-aware algorithm for tdma mac protocols in real-time wireless networks,” in _International Symposium on Industrial Embedded System (SIES)_. IEEE, 2010, pp. 156–165.
  * [81] S.-T. Cheng, M.-H. Tao, and C.-Y. Wang, “Adaptive channel switching for centralized mac protocols in multihop wireless networks,” _IEEE Transactions on Communications_ , vol. 58, no. 1, pp. 228–234, 2010.
  * [82] S. Ju and J. B. Evans, “Scalable cognitive routing protocol for mobile ad-hoc networks,” in _IEEE Global Communications Conference (GLOBECOM)_. IEEE, 2010, pp. 1–6.
  * [83] G. Naddafzadeh-Shirazi, P.-Y. Kong, and C.-K. Tham, “Distributed reinforcement learning frameworks for cooperative retransmission in wireless networks,” _IEEE Transactions on Vehicular Technology_ , vol. 59, no. 8, pp. 4157–4162, 2010.
  * [84] M. Mihaylov, Y.-A. Le Borgne, K. Tuyls, and A. Nowé, “Decentralised reinforcement learning for energy-efficient scheduling in wireless sensor networks,” _International Journal of Communication Networks and Distributed Systems_ , vol. 9, no. 3-4, pp. 207–224, 2012.
  * [85] Y. Chu, P. D. Mitchell, and D. Grace, “Aloha and q-learning based medium access control for wireless sensor networks,” in _International Symposium on Wireless Communication Systems (ISWCS)_. IEEE, 2012, pp. 511–515.
  * [86] N. Changuel, B. Sayadi, and M. Kieffer, “Online learning for qoe-based video streaming to mobile receivers,” in _IEEE GLOBECOM Workshops_. IEEE, 2012, pp. 1319–1324.
  * [87] S. Galzarano, A. Liotta, and G. Fortino, “Ql-mac: A q-learning based mac for wireless sensor networks,” in _International Conference on Algorithms and Architectures for Parallel Processing_. Springer, 2013, pp. 267–275.
  * [88] M. Rovcanin, E. De Poorter, I. Moerman, and P. Demeester, “A reinforcement learning based solution for cognitive network cooperation between co-located, heterogeneous wireless sensor networks,” _Ad Hoc Networks_ , vol. 17, pp. 98–113, 2014.
  * [89] A. Al-Saadi, R. Setchi, Y. Hicks, and S. M. Allen, “Multi-rate medium access protocol based on reinforcement learning,” in _IEEE International Conference on Systems, Man, and Cybernetics_. IEEE, 2014, pp. 2875–2880.
  * [90] Y. Chu, S. Kosunalp, P. D. Mitchell, D. Grace, and T. Clarke, “Application of reinforcement learning to medium access control for wireless sensor networks,” _Engineering Applications of Artificial Intelligence_ , vol. 46, pp. 23–32, 2015.
  * [91] S. Amuru, Y. Xiao, M. van der Schaar, and R. M. Buehrer, “To send or not to send-learning mac contention,” in _IEEE Global Communications Conference (GLOBECOM)_. IEEE, 2015, pp. 1–6.
  * [92] A. D. Shoaei, M. Derakhshani, S. Parsaeefard, and T. Le-Ngoc, “Learning-based hybrid tdma-csma mac protocol for virtualized 802.11 wlans,” in _IEEE Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC)_. IEEE, 2015, pp. 1861–1866.
  * [93] I. Kakalou, G. I. Papadimitriou, P. Nicopolitidis, P. G. Sarigiannidis, and M. S. Obaidat, “A reinforcement learning-based cognitive mac protocol,” in _IEEE International Conference on Communications (ICC)_. IEEE, 2015, pp. 5608–5613.
  * [94] N. Mastronarde, J. Modares, C. Wu, and J. Chakareski, “Reinforcement learning for energy-efficient delay-sensitive csma/ca scheduling,” in _IEEE Global Communications Conference (GLOBECOM)_. IEEE, 2016, pp. 1–7.
  * [95] M. Qiao, H. Zhao, S. Huang, L. Zhou, and S. Wang, “An intelligent mac protocol selection method based on machine learning in wireless sensor networks.” _KSII Transactions on Internet & Information Systems_, vol. 12, no. 11, 2018\.
  * [96] J. R. Cordeiro, D. F. Macedo, and L. F. Vieira, “Fs-mac: A flexible mac platform for wireless networks,” in _IEEE Wireless Communications and Networking Conference (WCNC)_. IEEE, 2018, pp. 1–6.
  * [97] J. Xu, Z. Dou, and L. Qi, “Multi-user dynamic spectrum access based on reinforcement learning,” in _International Conference on Graphics and Image Processing (ICGIP 2019)_ , vol. 11373. International Society for Optics and Photonics, 2020, p. 113732Y.
  * [98] A. Ranjha and G. Kaddoum, “Quasi-optimization of uplink power for enabling green urllc in mobile uav-assisted iot networks: A perturbation-based approach,” _IEEE Internet of Things Journal_ , vol. 8, no. 3, pp. 1674–1686, 2020.