#  $\mathsf{FBSDetector}$: Fake Base Station and Multi Step Attack Detection
in Cellular Networks using Machine Learning

Kazi Samin Mubasshir*  Purdue UniversityWest Lafayette, INUSA
<EMAIL_ADDRESS>,  Imtiaz Karim  Purdue UniversityWest Lafayette, INUSA
<EMAIL_ADDRESS>and  Elisa Bertino  Purdue UniversityWest Lafayette, INUSA
<EMAIL_ADDRESS>
###### Abstract.

Fake base stations (FBSes) pose a significant security threat by impersonating
legitimate base stations. Though efforts have been made to defeat this threat,
up to this day, the presence of FBSes and the multi-step attacks (MSAs)
stemming from them can lead to unauthorized surveillance, interception of
sensitive information, and disruption of network services for legitimate
users. Therefore, detecting these malicious entities is crucial to ensure the
security and reliability of cellular networks. Traditional detection methods
often rely on additional hardware, predefined rules, signal scanning, changing
protocol specifications, or cryptographic mechanisms that have limitations and
incur huge infrastructure costs in accurately identifying FBSes. In this
paper, we develop $\mathsf{FBSDetector}$–an effective and efficient detection
solution that can reliably detect FBSes and MSAs from layer-3 network traces
using machine learning (ML) at the user equipment (UE) side. To develop
$\mathsf{FBSDetector}$, we created $\mathsf{FBSAD}$ and $\mathsf{MSAD}$, the
_first-ever_ high-quality and large-scale datasets for training machine
learning models capable of detecting FBSes and MSAs. These datasets capture
the network traces in different real-world cellular network scenarios
(including mobility and different attacker capabilities) incorporating
legitimate base stations and FBSes. The combined network trace has a volume of
$\mathsf{6.6}$ GB containing $\mathsf{751963}$ packets. Our novel ML models,
specially designed to detect FBSes and MSAs, can effectively detect FBSes with
an accuracy of $\mathsf{92\%}$ and a false positive rate of $\mathsf{5.96\%}$
and recognize MSAs with an accuracy of $\mathsf{86\%}$ and a false positive
rate of $\mathsf{7.82\%}$. We deploy $\mathsf{FBSDetector}$ as a real-world
solution to protect end-users through an Android app and validate in a
controlled lab environment. Compared to the existing heuristic-based solutions
that fail to detect FBSes, $\mathsf{FBSDetector}$ can detect FBSes in the wild
in real time.

*Equal contribution. The student author’s name is given first.

##  1\. Introduction

The widespread adoption of cellular networks has brought about unprecedented
improvements in data rates, latency, and device connectivity, resulting in a
surge in the number of connected devices worldwide. In 2023, the number of
mobile devices is assessed at $\mathsf{16.8}$ billion having an estimated
$\mathsf{5.25}$ billion global users, marking a $\mathsf{4.9\%}$ annual
increase (oberlo-total-smartphones-in-2023, ), and the number of mobile
devices is expected to reach $\mathsf{18.22}$ billion by 2025 having
$\mathsf{5.67}$ billion users (statista-number-of-mobile-devices-worldwide-
from-2020-to-2025, ). Given the extensive usage and dependence on cellular
networks, they become desirable targets for malicious entities. These entities
try to disrupt cellular network services by exploiting different types of
vulnerabilities in cellular network protocols with the help of sophisticated
hardware. Of all the attacks and threat models targeting cellular networks,
Fake Base Stations (FBSes), a.k.a. rouge base stations, stingrays, and IMSI
catchers, are among the most widespread and represent a significant threat.
Due to the lack of authentication of initial broadcast messages as well as the
unprotected connection setup in the bootstrapping phase (PHOENIX, ; Insecure-
Connection-Root-of-All-Evil, ), adversaries can install FBSes that can lure
unsuspecting devices to connect to them and then launch sophisticated multi-
step attacks (MSAs) (DoLTEst, ; LTEFuzz, ; Hussain2018LTEInspectorAS, ;
altaf2016, ; 5GReasoner, ; Hussain2019PrivacyAT, ; erni2022adaptover, ).

Motivation. The threat posed by FBSes is not new and has been around for a
while, but they are still extensively used by attackers worldwide (FBS-News, ;
Stingray-THAI-NEWSROOM, ). The key motivations of our work are: (1) A
persistent problem. Despite numerous efforts, FBSes are still one of the most
persistent problems in cellular network security and can be deployed and used
for attacks in 5G cellular networks, despite the use of the encrypted
permanent ID SUCI compared to the unencrypted IMSI in 4G, which the UE sends
to the base station for authentication (chlosta20215g, ). Recent efforts have
introduced certificate-based solutions and digital
signatures(Singla2020ProtectingT4, ; Look-Before-You-Leap, ;
FBSSpecifications, ) to address such problems. Nonetheless, these proposals
are still in their infancy and would impose substantial overhead and require
changes to the specifications to be applicable in a real-world setting.
Moreover, the proposed certificate-based solutions make the design of roaming
difficult for cellular network providers. When a user travels from one place
to another place or from one country to another country, the different network
providers will have to share their secret keys for authentication, which is
very challenging considering the security aspects of the sharing process
(MobileAtlas, ). (2) Billions of unprotected devices.  The 3GPP is planning to
incorporate several defense mechanisms in the protocol to defend against FBSes
(FBSSpecifications, ) in future generations. However, these defense mechanisms
will still take several years to roll out in the specification and then to the
implementation. Currently, billions of devices worldwide are vulnerable to
FBS-based attacks, and billions of new devices released in the coming years
will be vulnerable to attacks until the new protocol is implemented. These
devices need to be secured through in-device solutions because replacing them
is impractical and would incur huge costs. (3) Impracticality and high cost of
existing defense mechanisms. Although efforts (Trustin5G, ; Karaçay2021, ;
Insecure-Connection-Root-of-All-Evil, ; Murat, ; CatchMeIfYouCan, ;
Cooper2021, ; Arslan2019, ; Hamad2019ICCAIS, ; Huang2018IdentifyingTF, ;
Onetraceisallittakes, ; NakarmiML2022, ; Li2017FBSRadarUF, ; Darshak, ;
AIMSICD, ; BARON, ; PHOENIX, ; SnoopSnitch, ) have been undertaken to detect
and take down FBSes, they suffer from at least one of the following
limitations: (A) Heuristic (Darshak, ; AIMSICD, ; SnoopSnitch, ) and signature
(PHOENIX, ) based solutions fail to adapt to the detection of ever-evolving
attacks. (B) Some solutions depend on crowd-sourced data (Li2017FBSRadarUF, ),
which is impractical to scale up to a system that has to protect billions of
devices. (C) Some detection solutions (CatchMeIfYouCan, ; Cooper2021, ; Murat,
; OVERWATCH, ; SeaGlass, ) require installing expensive hardware; in most
cases, they are proprietary. For example, CellDAM (CellDAM, ) requires a
separate companion node near the UEs to capture the signaling messages. This
is not a practical solution for protecting billions of devices. (D) Lower
layer based solutions (Arslan2019, ; Hamad2019ICCAIS, ;
Huang2018IdentifyingTF, ; Karaçay2021, ; NakarmiML2022, ; Trustin5G, ;
Onetraceisallittakes, ; RF-Fingerprinting-2023, ) cannot detect sophisticated
FBSes, and are not inherently able to detect MSAs.

Recently, both Google and Apple have taken new ways to defeat FBSes (New-Ways-
to-Defeat-Cell-Site-Simulators, ). Though promising, these measures have
little impact on a knowledgeable and well-equipped adversary. Therefore, it is
essential to design effective and efficient solutions able to detect FBSes and
MSAs. In this paper, we aim to address such a need by developing a low-
overhead, no-cost, and in-device solution that can effectively detect FBSes
and MSAs that use FBSes in their threat model, from the network traces in the
UEs.

A practical solution. Machine learning (ML) can be a practical in-device
solution to address all the existing problems in detecting and defending
against the FBSes and MSAs. On a high level, an ML-based solution has the
following benefits:  ❶ It can protect all the existing end-user devices
vulnerable to the attacks.  ❷ No change is required in the protocol. Using the
network traces, especially the higher layer (layer 3) traces, the ML
algorithms can determine if there are any FBSes in the network and recognize
MSAs.  ❸ No additional hardware is required.  ❹ ML algorithms add little
overhead regarding memory and power consumption.  ❺ As the attack patterns are
similar worldwide, ML algorithms can detect FBSes and MSAs anywhere in the
world and thus can support roaming of the device they are deployed in.

A critical issue when applying ML techniques to FBS and MSA detection is which
features to use. Most previous solutions rely on features extracted from the
physical layer traces. However, as mentioned earlier, such approaches have
several limitations. Our insight about this issue is that features extracted
from traces at the layer-3 protocols are much more informative - which is
critical for being able to detect not only FBSes but also detect and classify
MSAs, especially when dealing with sophisticated attackers.

Challenges. The design of an ML-based system for the detection of FBSes,
however, requires addressing several major challenges: (1) Acquiring a
comprehensive and high-quality dataset encompassing a wide range of real-world
cellular network scenarios. (2) Incorporating the surrounding context into
consideration. Unlike traditional methods where packets can be inherently
malicious, in this problem, the true nature of a packet hinges on “when” and
“in which context” it was sent. (3) Capturing, representing, and learning the
unique characteristics that define these MSAs. The actual challenge lies in
learning these complex attack signatures. This task requires the deployment of
sophisticated ML algorithms that not only recognize known attack patterns but
also adapt to evolving attacks. (4) Combining the predictions of layer-3
protocols. Layer-3, also known as the network layer, has two key protocols:
(1) NAS (Non-Access Stratum) (NAS, ) and (2) RRC (Radio Resource Control)
(RRC, ). These protocols operate within the control plane, each serving
distinct purposes in facilitating communication between User Equipment (UE)
and the network infrastructure. To have a unified detection and improve
robustness, the predictions made separately for the two protocols need to be
combined. (5) Enabling real-time detection of the attacks. The framework needs
to be an in-device solution that captures, processes and analyzes incoming
packets promptly to detect the presence of the attacks effectively.

Our approach. To detect FBSes effectively, in this paper, we present the
design, implementation, and deployment of $\mathsf{FBSDetector}$, an ML-based
framework for FBS detection and MSA recognition. As it is illegal to create
FBSes in public areas and there are no publicly available datasets of FBS and
MSA traces, we create $\mathsf{FBSAD}$ and $\mathsf{MSAD}$, the first
comprehensive and high-quality FBS and MSA datasets, respectively. To achieve
this, we utilize different facilities at POWDER (powder, ). POWDER is a city-
scale and end-to-end software-defined platform to support mobile and wireless
research. The dataset created using POWDER is analogous to real-world
datasets. This is ensured by including over-the-air actual packets transmitted
within its spectrum between actual devices instead of simulated wire
transmissions (powder-usecase1, ; powder-usecase2-nexran, ; powder-
usecase3-rf-propagation, ; powder-usecase4-WiMatch, ; powder-usecase5-ZCNET,
). $\mathsf{FBSAD}$ and $\mathsf{MSAD}$ incorporate different networking
scenarios (static and mobile endpoints) and different attacker capabilities
(from less sophisticated to more sophisticated). To tackle the second
challenge, that is, incorporating the surrounding context into detection, we
design a Sequential-LSTM model that takes sequence length as a hyperparameter
and trains on chunks of packets. This ensues from our intuition that the
length of the sequence of FBS-generated messages follows a specific
distribution (see Section 6.1). For MSA detection, we use graph
learning–derived from our intuition that all the MSAs follow a specific
pattern (see Section 4.2.2), and that to recognize an MSA successfully, it is
necessary to capture these patterns. We construct a directed graph from the
packets to capture the patterns and train a graph learning model on these
graphs to learn the patterns. Moreover, through using maximum overlapping sub-
graphs, we can detect unseen attacks and recognize the MSAs, even if they
evolve or the attackers try to evade detection by re-shaping traffic (see
Section 4.2.2). To obtain a unified prediction we combine predictions made on
NAS and RRC packets using a weighted confidence-based fusion method. This
method of combining predictions enhances detection accuracy beyond individual
predictions. For the deployment of $\mathsf{FBSDetector}$, we create a mobile
app that can scan the network and analyze the packet traces by running the
pre-trained models on the device to detect FBSes and MSAs effectively in real
time.

Experimental results. The unprocessed $\mathsf{FBSAD}$ and $\mathsf{MSAD}$
datasets have a combined size of $\mathsf{6.6}$ GB. After the initial
filtering, we have $\mathsf{751963}$ packets in total. Trained on this
combined dataset, our experiments show that our Sequential-LSTM model can
detect FBS with $\mathsf{92\%}$ accuracy and a false positive rate of
$\mathsf{5.96\%}$. Similarly, our graph learning model can detect $\mathsf{9}$
MSAs with $\mathsf{86\%}$ accuracy and a false positive rate of
$\mathsf{7.82\%}$. Our experiments also show that combining NAS and RRC
predictions using weighted confidence-based fusion method improves the
performance beyond the individual models trained on NAS and RRC layers
separately by $\mathsf{1\sim 2\%}$. To validate $\mathsf{FBSDetector}$’s
fidelity and evaluate its performance, memory and power consumption in real-
world scenarios, we instantiate an Android application for 4G UEs and set up
FBSes and MSAs in a controlled lab environment. Using our lab setup, we spawn
an FBS and run experiments with different FBS detection and MSA recognition
scenarios. The experimental results show that $\mathsf{FBSDetector}$ can
detect FBSes in all the tested scenarios using an average $\mathsf{835}$ KB of
memory and less than $\mathsf{2}$ mW of power.

Contributions. This paper makes the following contributions:

  * •

We develop a new framework–$\mathsf{FBSDetector}$ to detect FBSes and MSAs
from network traces using ML. For this, we create $\mathsf{FBSAD}$ and
$\mathsf{MSAD}$, the _first-ever_ large-scale, high-quality, real-world
datasets containing FBS and MSA traces in different scenarios.

  * •

We design a Sequential-LSTM model for the FBS detection task, which improves
the detection accuracy and lowers the false positive rates than other state-
of-the-art models. For MSAs, we innovate by converting the attack signatures
to graphs and using a graph-based learning approach to detect the attacks.
Graph learning models perform better than any other state-of-the-art model in
recognizing MSAs. Moreover, the MSAs, even if they evolve, and unseen and
reshaped attacks can be detected in this approach by using maximum overlapping
sub-graphs.

  * •

We deploy the solution in an Android app and validate its performance in real
environments by creating a testbed in our controlled lab setup. Compared to
the available end-user FBS detection solutions, which fail to detect FBSes and
MSAs, our approach effectively detects FBSes and MSAs in all the tested real-
world scenarios.

Open-source. We will open-source the datasets, our models, and the Android app
for further research and help end-users protect themselves from FBS-related
attacks.

##  2\. Background

In this section, we introduce relevant background about 4G, FBSes, MSAs and
POWDER–the platform we use for dataset generation.

###  2.1. 4G Cellular Networks

In a 4G network, cellular devices are called User Equipments (UE). The core
network is called the Evolved Packet Core (EPC). Geographic locations are
partitioned into hexagonal cell areas, each of which is serviced by a
designated base station (eNodeB), which enables connectivity of UEs in that
cell to the EPC. The Mobility Management Entity (MME) manages the connectivity
and mobility of UEs in a particular tracking area (a set of cell areas).

Non-Access Stratum (NAS). In 4G, the Non-Access Stratum (NAS) (NAS, ) protocol
is a layer-3 (Network Layer) protocol specified by 3GPP that serves as a
functional layer between the core network and the UEs. Its primary role is to
manage the communication sessions and seamlessly maintain the connections with
the UE, even when the UE roams.

Radio Resource Control (RRC). The Radio Resource Control (RRC) (RRC, )
protocol is another layer-3 protocol used between the UEs and the base
stations. The major functions of the RRC protocol include connection
establishment and release functions, broadcast of system information, radio
bearer establishment, reconfiguration and release, and paging notification and
release.

Figure 1. Operating principles of a typical FBS. Compared to a legitimate base
station, an FBS usually possesses a smaller signal coverage area but a higher
signal strength.

###  2.2. Fake Base Station and Multi-Step Attacks

A Fake Base Station (FBS) is an unauthorized device that impersonates a
legitimate base station within a cellular network (shown in Figure 1.b). FBSes
typically consist of a radio transceiver capable of broadcasting signals on
the same frequencies legitimate base stations use. By emitting these signals,
an FBS creates a cell or coverage area, attracting nearby mobile devices to
connect to it.

When a mobile device connects to an FBS, it establishes a communication link
with the FBS instead of the legitimate network. This enables the attacker
operating the FBS to carry out various malicious activities, including call
interception, SMS interception, location tracking, man-in-the-middle attacks,
and denial of service (erni2022adaptover, ; kotuliak2022ltrack, ; altaf2016, ;
Hussain2019PrivacyAT, ; LTEFuzz, ; Hussain2018LTEInspectorAS, ; 5GReasoner, ).
Some FBSes can mimic the network name (SSID) and encryption protocols used by
legitimate networks, making it difficult for users to differentiate between
fake and genuine networks. This can trick users into unknowingly connecting to
the FBS, making them susceptible to attacks. The system architecture of FBSes
has evolved over several generations, where each generation uses different
hardware and communication processes. Utilizing a FBS as a foothold, an
attacker can also carry out several coordinated steps to achieve a malicious
objective. These Multi-Step Attacks (MSAs) can compromise cellular
communication’s integrity (DoLTEst, ; LTEFuzz, ), privacy
(Hussain2018LTEInspectorAS, ; altaf2016, ; 5GReasoner, ; Hussain2019PrivacyAT,
), and availability (Hussain2018LTEInspectorAS, ; erni2022adaptover, ;
altaf2016, ; 5GReasoner, ) and usually take several specific well-defined
attack steps.

Hardware and Software. A FBS is generally made up of two components: (1) a
wireless transceiver and (2) a computing unit (shown in Figure 1.a). The
components are portable and require a relatively small amount of electrical
power, making it easy to accommodate them in a van, a minibus, the trunk of a
car, or even a backpack. The mobility of modern FBSes makes it difficult to
localize and take them down. The wireless transceiver is the core component of
an FBS. It can simulate most of the base-station-to-cellphone functions of a
legitimate base station but not the base-station-to-carrier functions (i.e.,
it cannot forward voice, SMS, and data packets to a mobile carrier’s network)
due to the absence of network authorization and the presence of security
mechanisms. The computing unit, in most cases a laptop (in more modern cases a
Raspberry Pi, or an Intel NUC), is usually connected to the wireless
transceiver via a USB cable, through which it controls the transceiver, e.g.,
tuning the radio frequency, adjusting the signal strength, setting the base
station ID, and other parameters. A cellphone searches for nearby, legitimate
base stations and records their wireless parameters. Based on this
information, the attacker can reconfigure the parameters of the FBS to mimic
legitimate base stations to maximize the number of affected users.

Communication Process. To interrupt the existing connections between nearby
user devices and legitimate base stations, the attacker would adjust the
signal strength of the FBS to guarantee that it offers a much higher signal
strength than the legitimate base station. Once the operator has properly
configured the FBS, usually the following sequence of events begins to occur,
culminating in messages being sent to all users nearby:  ① The FBS broadcasts
its system information using the configured radio frequency.  ② When a UE
receives the system information of the FBS, it detects it as a new base
station with a higher signal strength and sends a tracking area update request
to the FBS. In other cases, the FBS can send a paging request to trigger the
service procedure to establish the communication.  ③ Upon receiving the
tracking area update request, the FBS first usually sends an identity request
to the UE to acquire its IMSI information and then sends another identity
request to acquire its IMEI information.  ④ Subsequently, the FBS sends
different messages to the UE. This action can be repeated multiple times.  ⑤
When the FBS completes sending messages, it cuts off the cellular connections
with its UEs by lowering its signal strength or shutting down the signal.
After that, the affected UEs may or may not automatically re-connect to a
legitimate BS (some UE’s require manual restarting to connect back). A
simplified diagram of this communication process is shown in Figure 2.

Figure 2. Usual communication process of FBSes

###  2.3. POWDER

POWDER (Platform for Open Wireless Data-driven Experimental Research) (powder,
) is a cityscale, remotely accessible, end-to-end software-defined platform
funded by NSF to support mobile and wireless research and provides advances in
scale, realism, diversity, flexibility, and access.

Fidelity of POWDER to the real-world. POWDER’s fidelity extends beyond
conventional simulation due to the following reasons: (1) POWDER provides a
dedicated frequency band and real devices (powder, ). This access to real
devices while maintaining scalability and mobility makes POWDER equivalent to
real-world testbeds. (2) POWDER’s fidelity is proven through rigorous testing
and evaluation, demonstrating that solutions developed using POWDER, which
mirrors the complexity and challenges found in real network environments,
perform effectively well in real-world scenarios (powder-usecase1, ; powder-
usecase2-nexran, ; powder-usecase3-rf-propagation, ; powder-usecase4-WiMatch,
; powder-usecase5-ZCNET, ). Because of incorporating over-the-air actual
packets in a dedicated spectrum instead of simulated wire transmissions,
datasets created using POWDER are analogous to real-world datasets. A detailed
overview of POWDER is given in the appendix section C.

##  3\. Overview

In this section, we discuss the scope, threat model, challenges, and
requirements of $\mathsf{FBSDetector}$.

###  3.1. Scope

The scope of $\mathsf{FBSDetector}$ is detecting FBSes and MSAs in the context
of 4G cellular networks. There are two significant reasons for this. _First,_
because of the extensive infrastructure already in place, 4G networks are
widely accessible to a larger portion of the population. As of 2023, 4G
adoption stands at 60% among 8.5 billion mobile connections
(4g-5g-subscribers-march-2022-quarterly-update, ;
ericsson_news_Mobile_subscriptions_outlook, ; telegeography, ). 4G adoption is
predicted to stay above 50% until 2027, and in 2029, 5G is expected to
overtake 4G (The-Mobile-Economy-2023, ; statista, ). Therefore, until 2027,
many end-user devices using 4G are at risk of FBS attacks. _Second,_ the
platform we use for real-world dataset generation for FBS and MSAs (i.e.,
POWDER) supports 4G functionalities and real-world experiments can be run in
POWDER for different scenarios only in 4G. POWDER currently does not support
all the 5G functionalities (for instance, handover). Our discussion with the
POWDER team revealed that support for all 5G functionalities is in their
plans. Nonetheless, extending $\mathsf{FBSDetector}$ to support 5G would
require minimal effort once 5G support is available in POWDER because the
high-level protocols of 4G and 5G are very similar. Therefore, the design of
$\mathsf{FBSDetector}$ will remain the same.

###  3.2. Threat Model

This section outlines the key aspects of the threat model in the context of
FBSes and MSAs.

FBS Deployment. The adversary can deploy FBSes in proximity to legitimate base
stations. These FBSes may range from small to more sophisticated setups with
varying transmission power.

Impersonation. FBSes can impersonate legitimate base stations by broadcasting
signals that mimic the network identifiers (e.g., SSID) and encryption
protocols used by legitimate base stations. However, the FBS does not know any
cryptographic keys of the communication. By impersonating trusted networks,
attackers can deceive mobile devices into connecting to FBSes.

MSA. Once UEs connect to the FBSes, adversaries can intercept and manipulate
communication traffic. Adversaries may exploit vulnerabilities in the
communication protocols by sending different out-of-sequence messages to
perform different MSAs. Furthermore, they can change the MSA or re-shape
attack traffic to evade detection.

Detection Evasion. Attackers may employ various techniques to evade detection.
This includes rapidly changing the parameters of the FBSes, adjusting
transmission power, or adopting sophisticated obfuscation methods to mask
their activities.

###  3.3. Challenges

The challenges in designing our ML-based framework for detecting FBSes and
MSAs are:

C1. Dataset Availability and Quality. Acquiring a comprehensive and high-
quality dataset that encompasses a wide range of real-world cellular network
scenarios is a significant challenge. The dataset must include instances of
legitimate base stations, FBSes, and execution traces of different MSAs.
Currently, there is no such dataset publicly available. There are several
reasons behind this: (1) The law prohibits the deployment of FBSes in public
areas. If someone wants to deploy it for research and experimentation, they
must do it in a controlled RF environment. (2) Incorporating different real-
world cellular network scenarios, such as handovers and mobility, is a
difficult task and would require a lot of specialized hardware and other
equipment and facilities.

C2. Detecting FBSes from Packet Traces. Unlike traditional methods where
packets themselves are inherently malicious, in the case of detecting FBSes,
the true nature of a packet hinges on which context a packet was sent. For
instance, the same packet with all the same contents can be sent by a base
station or an FBS. Moreover, a critical constraint is maintaining the original
sequence of packets as they were sent and received throughout the process to
preserve the context information.

C3. Detecting MSAs. Recognizing MSAs from packet traces is even more
challenging than FBS detection. These attacks have unique characteristics that
define them. Moreover, MSAs often exhibit complex, evolving patterns that
require careful observation to distinguish them from legitimate traffic. These
attacks are notorious for their adaptability, constantly changing to evade
detection. Consequently, our approach must evolve alongside these threats. We
must capture these attack characteristics and represent them effectively in a
structured data format.

C4. Combining NAS and RRC Predictions. Training our models separately on NAS
and RRC layer packets is a necessary step due to their distinct features and
characteristics. However, a challenge arises when we must consolidate these
separate model predictions into a unified model. The challenge lies in
harmonizing these diverse sets of predictions, ensuring that our final model
can effectively capture the information of both layers while providing robust,
accurate predictions.

C5. Real-Time Detection. Enabling real-time detection of FBSes and MSAs is a
challenge due to the large volume and velocity of network traffic. The
framework needs to be an in-device solution that captures, processes and
analyzes incoming packet traces promptly to detect the presence of FBSes and
MSAs effectively.

###  3.4. Proposed Solution

S1. We use different facilities available at POWDER (powder, ) to create
$\mathsf{FBSAD}$ and $\mathsf{MSAD}$, real-world datasets to detect FBSes and
MSAs. Designing different networking scenarios, we incorporate legitimate base
stations and FBSes and collect data. We also collect network traces of
different MSAs that use FBSes in their threat model. Different adversaries are
incorporated for both FBSes and MSAs, from less sophisticated to more
sophisticated, with the ability to clone legitimate base stations, change
signatures, and turn on and off to evade detection. The dataset is then
processed to make it appropriate for training different ML models. This
includes protocol filtering, feature extraction, feature alignment, and
replacing missing values.

S2. The lengths of the sequence of packets generated by FBSes follow a
specific distribution (shown in Figure 7(a) and 7(b)). Based on this
observation, we design a Sequential-LSTM model that takes sequence length as a
hyperparameter and trains on chunks of packets. Another essential objective
this design helps us achieve is incorporating the surrounding context into
consideration. This approach for FBS detection improves the detection accuracy
and lowers the false positive rates more than any state-of-the-art model.

S3. We observe that each MSA shows a unique pattern if they are represented as
a directed graph (see Figure 5 in Section 4.2.2). Based on this observation,
we devise an innovative solution centered around graph data structures and
learning techniques. We transform the packet traces into directed graphs. This
graph representation captures the relationships inherent in the trace, which
is ideal for detecting MSAs with complex and evolving patterns. We train a
graph learning model specialized in learning complex patterns within the graph
and learn the patterns of the MSAs. With the trained model, we can recognize
the MSAs even if they evolve and unseen and reshaped attacks in our dataset
with maximum overlapping sub-graphs.

S4. To combine the model predictions for RRC and NAS packets in FBS and MSA
detection, we design a weighted confidence-based fusion method, a widely used
technique in the multi-sensor information fusion field (ZHANG2022202, ;
Tang2023, ). The weights are assigned to the best model of each layer based on
the model performance. This method offers a robust means to blend the
predictions for NAS and RRC layer packets, facilitating the creation of a
unified and accurate predictive framework that captures the strengths of each
layer.

S5. The development of a mobile app emerges as a pragmatic solution for
deploying the $\mathsf{FBSDetector}$ for real-time detection. Such a dedicated
app would serve as an in-device solution, capable of swiftly capturing,
processing, and analyzing incoming packet traces with on-device ML models,
specifically tailored for detecting FBSes and MSAs. Regular model updates
would ensure adaptability to threats and new and unseen attacks.

##  4\. Detailed Design

In this section, we discuss the detailed design of $\mathsf{FBSDetector}$ (see
Figure 3 for an overview). On a high-level, the design of
$\mathsf{FBSDetector}$ is divided into three components: (1) Dataset
Construction, (2) ML Framework, and (3) Deployment.

Figure 3. Overview of FBS-Detector

###  4.1. Dataset Construction

Our dataset construction process is described below.

####  4.1.1. Dataset Generation

We create cellular networks in POWDER incorporating legitimate base stations,
FBSes, and MSAs, run experiments, and capture the packets from all the
cellular network components to generate the dataset.

Mobility.  One real-world phenomenon in cellular networks is the mobility of
the UEs. Signal strengths of base stations received at UEs moving from one
place to another vary, causing the UEs to be handed over from one base station
to another. Incorporating this scenario in the dataset is important;
otherwise, a benign handover due to mobility might be interpreted as
malicious. With the help of mobile endpoints available at POWDER, we
incorporate this scenario into our dataset.

Attacker Ability.  The attackers we consider have a diverse set of abilities,
from less sophisticated to more sophisticated. Based on their abilities, we
can rank them in five levels, level 0 being the least sophisticated and level
4 being the most sophisticated.

  * •

Level 0: Attackers only set up FBSes naively with a high signal strength.

  * •

Level 1: Attackers set up FBSes with an optimal signal strength sufficient to
trigger a handover in the UEs.

  * •

Level 2: Attackers can clone all the parameters of a legitimate base station
and impersonate the legitimate base station.

  * •

Level 3: Attackers can carry out usual MSAs.

  * •

Level 4: Attackers can change the typical attack signatures that the FBSes and
MSAs follow and reshape traffic. Attacks can be reshaped by changing the
packet orders and packet fields. It is known that there are typical defenses
deployed and the attackers actively aim to evade them.

MSA Data Generation.  To create $\mathsf{MSAD}$, we chose several attacks that
are executed in multiple steps and use FBS in their threat model. As shown in
Table 1, we have selected $\mathsf{9}$ attacks, which are representative of
the most popular MSAs. These attacks cover a wide range of practical threats,
including DoS, privacy leakage, and downgrade, which can be quickly launched
to compromise practical networks and devices severely. We implement and
execute these attacks in the cellular networks in POWDER and incorporate
instances of each attack in $\mathsf{MSAD}$.

Layer | Attack | Ref
---|---|---
RRC | Energy Depletion Attack | (Hussain2018LTEInspectorAS, )
Paging Channel Hijacking Attack | (Hussain2018LTEInspectorAS, )
Panic Attack | (Hussain2018LTEInspectorAS, )
Stealthy kicking-off Attack | (Hussain2018LTEInspectorAS, )
Incarceration Attack | (5GReasoner, )
NAS | Numb Attack | (Hussain2018LTEInspectorAS, )
Detach/Downgrade Attack | (Hussain2018LTEInspectorAS, )
Uplink NAS Counter Desync Attack | (5GReasoner, )
Downgrade Attacks | (BiddingDownAttacks, )
Table 1. Multi-Step Attacks using FBS

####  4.1.2. Dataset Preprocessing

In order to make $\mathsf{FBSAD}$ and $\mathsf{MSAD}$ suitable for training ML
models, we pre-process them in several steps.

Protocol filtering, field extraction and alignment. Each packet in the network
trace contains multiple protocol information. We focus on only NAS and RRC
data and use protocol filtering to isolate the relevant packets precisely. We
further extract the values of fields associated with these packets. A
challenge is the presence of varying field counts among different packets; to
address this, we align the packets based on their shared field values. This
process ensures consistency in our dataset, enabling uniformity in subsequent
analyses and modeling.

Handling missing values. The alignment of packets based on shared field
values, when specific fields are disjoint across packets, results in missing
values within the dataset– a scenario that cannot be resolved using
conventional methods like mean or median imputation, as packet features lack a
typical distribution. To tackle this issue and ensure data completeness, we
replace the missing values with a flag value of $-1$ to indicate their
absence. This uniform marker across the dataset is a pragmatic solution,
allowing us to retain valuable features without introducing bias or
compromising data quality.

Encoding the categorical values.  In our dataset, numerous fields take on
categorical values, which hinders the model’s understanding. We thus encode
the categorical fields to numerical representations to make them suitable for
training ML models.

####  4.1.3. Dataset Labelling

The datasets are labeled according to their reason for generation. Formally,
for a packet $\mathcal{P}$ we define the following formulation:

Let,

$\mathsf{FBSAD}\coloneqq<X_{\mathsf{FBSAD}},Y_{\mathsf{FBSAD}}>$

where $X_{\mathsf{FBSAD}}$ are the packet features, $Y_{\mathsf{FBSAD}}$ are
the packet labels and,

$\begin{split}i\in\\{1,n_{\mathsf{FBSAD}}\\},where\;n_{\mathsf{FBSAD}}\
=|X_{\mathsf{FBSAD}}|\\\ {\overrightarrow{x}_{\mathsf{FBSAD}}}^{(i)}\in
X_{\mathsf{FBSAD}},{\overrightarrow{y}_{\mathsf{FBSAD}}}^{(i)}\in
Y_{\mathsf{FBSAD}}\end{split}$

then we can label each packet $\overrightarrow{x}_{\mathsf{FBSAD}}^{(i)}$ as

${\overrightarrow{y}_{\mathsf{FBSAD}}}^{(i)}=\begin{cases}0&\text{if
${\overrightarrow{x}_{\mathsf{FBSAD}}}^{(i)}$ is a benign packet}\\\
1&\text{if ${\overrightarrow{x}_{\mathsf{FBSAD}}}^{(i)}$ is generated from
FBS}\end{cases}$

For $\mathsf{MSAD}$, we define
$\mathcal{A}\coloneqq\\{attack_{1},attack_{2},\cdots,attack_{k}\\}$ as the set
of MSAs detected by $\mathsf{FBSDetector}$ where $k$ is the number of MSAs
that can be detected and
$\mathcal{L}_{\mathcal{A}}\coloneqq\\{0,1,2,\cdots,k\\}$ as the set of labels
for the MSAs. Let,

$\mathsf{MSAD}\coloneqq<X_{\mathsf{MSAD}},Y_{\mathsf{MSAD}}>$

where $X_{\mathsf{MSAD}}$ are the packet features, $Y_{\mathsf{MSAD}}$ are the
packet labels and

$\begin{split}i\in\\{1,n_{\mathsf{MSAD}}\\},where\;n_{\mathsf{MSAD}}\
=|X_{\mathsf{MSAD}}|\\\ {\overrightarrow{x}_{\mathsf{MSAD}}}^{(i)}\in
X_{\mathsf{MSAD}},{\overrightarrow{y}_{\mathsf{MSAD}}}^{(i)}\in
Y_{\mathsf{MSAD}}\end{split}$

then we can label each packet $\overrightarrow{x}_{\mathsf{MSAD}}^{(i)}$ as

${\overrightarrow{y}_{\mathsf{MSAD}}}^{(i)}=\begin{cases}0&\text{if
${\overrightarrow{x}_{\mathsf{MSAD}}}^{(i)}$ is a benign packet}\\\
\mathcal{L}_{\mathcal{A}}[attack_{j}]&\text{if
${\overrightarrow{x}_{\mathsf{MSAD}}}^{(i)}$ is generated}\\\ &\text{by
$attack_{j}\in\mathcal{A}$}\end{cases}$

###  4.2. Machine Learning Framework

In what follows, we detail the ML framework used in $\mathsf{FBSDetector}$.

####  4.2.1. FBS Detection

FBSes generate a sequence of messages, and detecting a single packet often
leads to classification errors due to lack of context. Following these
observations, we design a Sequential-LSTM model that takes the length of the
sequence into account (shown in Algorithm 1). The algorithm essentially
divides the dataset into chunks of sequences using sequence length
$(len_{seq})$ as a hyperparameter and returns a Seq-LSTM model trained on
these sequences. This approach can help us learn better from the context of a
chunk with the information about an FBS communication, improve the detection
accuracy, and reduce the false positive rate.

Algorithm 1 FBS and MSA Learning Algorithms

1:Input: Hyperparameter: $len_{seq}$,

2:Labeled dataset: $\mathsf{FBSAD}$, $\mathsf{MSAD}$

3:Output: Seq-LSTM model, Graph Model (GM)

4:procedure FBS Training

5: Divide $\mathsf{FBSAD}$ into $\frac{|\mathsf{FBSAD}|}{len_{seq}}$ chunks
$C$

6: for $c$ in $C$ do

7: Train the Seq-LSTM model on $c$

8: end for

9:end procedure

10:procedure Graph Learning

11: Variables: Graph $G(V,E)$

12: Create start node $V_{1}$ with the first packet $p_{1}$

13: for each subsequent packet $p_{i}$ in $\mathsf{MSAD}$ do

14: if $V_{p}$ not in $G$ then

15: Create a node $V_{p}$

16: end if

17: Add an incoming edge $E_{p}$ from $V_{p-1}$ to $V_{p}$

18: Label $E_{p}$ with $L_{p}$, the Label for Packet $p$

19: end for

20: $GM=\text{train}(G)$

21:end procedure

22:return Seq-LSTM, GM

####  4.2.2. MSA Recognition

MSAs can be uniquely represented as directed graphs and detected using graph
learning. We describe the steps to create a directed graph from the traffic
dataset and graph learning in Algorithm 1. The algorithm constructs a directed
graph from the packets - each node denoting a packet, each packet having a
directed edge towards the next packet. Edges are labeled with their reason for
generation, the same as the packet label, generated as part of a benign flow
or in the path of an MSA. Lastly, a graph learning model learns and
generalizes this knowledge from the graphs, and the model is finally returned.
Recognizing MSAs using graph learning approaches can be formally represented
as follows. Let $G=(V,E)$ represent the graph constructed from the packets,
where each node $V$ denotes a packet. $E$ represents the directed edges
between nodes in the flow graph. An MSA is recognized by identifying a
specific path $P(G)$ corresponding to $\mathcal{A}$ within the graph $G$.
These paths represent the packets’ sequences that follow an MSA pattern. If
there is an evolving and unseen attack, the attack would deviate from path
$P(G)$ and follow a different path $P^{\prime}(G)$. Due to the nature of the
vulnerabilities exploited by the attackers, $P^{\prime}(G)$ and $P(G)$ will
not be completely edge-disjoint and will have overlaps. From these overlaps,
we can detect evolving, unseen and reshaped attacks.

MSA Detection Example.  To further illustrate the process of MSA detection, we
discuss the paging channel hijacking attack, its attack flow, and how it is
converted to a graph to be utilized by graph learning algorithms.

Paging Channel Hijacking: To hijack the paging channel of the victim UE and
the legitimate base station, the FBS broadcasts fake empty paging message in
the shared paging channel with higher signal power. Upon receiving the Paging
request, the victim UE detaches itself from the legitimate base station and
connects to the FBS. This attack diagram is shown in Figure 4 and Figure 5
shows the graph generated from this attack. The communication starts when a UE
sends a AttachRequest message to the legitimate base station, which is the
starting node of the graph. An incoming edge is added to the node that
represents the message for the subsequent messages. The graph enters into an
attack sequence when a paging(empty) message is sent after the AttachComplete
message. Communication terminates when a DetachRequest message is sent; this
is the terminating node of the graph.

Figure 4. Paging Channel Hijacking Attack Diagram Figure 5. Graph of paging
channel hijacking attack

####  4.2.3. Combining NAS and RRC Predictions

To combine the predictions from the NAS and RRC layer models, we leverage the
Dempster–Shafer theory (DST) (Dempster–Shafer-theory, ) to facilitate the
fusion of predictions. We employ a weighted confidence-based fusion method,
where weights are assigned to the best model of each layer based on the model
performance. Let $W_{NAS}$ and $W_{RRC}$ represent the weights assigned to the
best models of NAS and RRC layer, $P=\\{P_{NAS},P_{RRC}\\}$ represent the
predictions made by those models, and $P_{W}$ is the final prediction. We
assign weights $W_{NAS}$ and $W_{RRC}$ proportional to their support scores on
the inference. The more confident model among the two models receives a higher
weight, indicating a more significant influence on the combined prediction.
Mathematically, this is expressed as:

$W_{NAS}\propto\text{Support\\_Score($P_{NAS}$)}\text{ and
}W_{RRC}\propto\text{Support\\_Score($P_{RRC}$)}$

When there’s a disagreement between the models, the Dempster–Shafer theory is
applied, and the prediction with higher confidence gets priority. This process
is formalized as:

$\text{$P_{W}$}=\begin{cases}\text{$P_{NAS}$}&\text{if
}\text{$P_{NAS}$}=\text{$P_{RRC}$}\\\ \text{$P_{NAS}$}&\text{if
}\text{$P_{NAS}$}\neq\text{$P_{RRC}$}\text{ and
}\text{$W_{NAS}$}>\text{$W_{RRC}$}\\\ \text{$P_{RRC}$}&\text{if
}\text{$P_{NAS}$}\neq\text{$P_{RRC}$}\text{ and
}\text{$W_{NAS}$}<\text{$W_{RRC}$}\end{cases}$

Combining the predictions in this way yields a better detection accuracy than
the individual predictions.

##  5\. Implementation

Now, we discuss the implementation details of each component of
$\mathsf{FBSDetector}$. We formulate two distinct models: one designed to
detect FBSes and another specifically tailored to recognize MSAs.
Implementation efforts of $\mathsf{FBSDetector}$ are summarized in Table 2.

Task | Framework | Lang |  Libraries Used |  LoC
---|---|---|---|---
Handover capability in srsUE | srsRAN | C |  |  $\mathsf{316}$
MSAs | srsRAN | C |  |  $\mathsf{542}$
Data Processing |  | Python |  tshark, scikit-learn |  $\mathsf{72}$
Training ML Models |  | Python |  scikit-learn, TensorFlow |  $\mathsf{40}$
Sequential LSTM |  | Python |  scikit-learn |  $\mathsf{42}$
Graph Learning Pipeline |  | Python |  networkx, pytorch-geometric |  $\mathsf{70}$
App Development | Flutter | Dart |  mobileinsight-libs, tensorflow-lite |  $\mathsf{290}$
Table 2. Implementation efforts

###  5.1. Dataset Construction

Incorporation of Fake Base Stations.  To incorporate FBSes, we create (i) a
legitimate core network and base station pair and (ii) a fake core network and
base station pair in POWDER. The legitimate base station serves UEs that the
FBS can attack by spawning near it with higher signal strength. The core
networks use Open5GS (Open5GS, ) to support handover, and both the base
stations and the UE use srsRAN (srsRAN, ). Following our threat model, the FBS
core network does not know the cryptographic keys. To incorporate less
sophisticated attackers, we only set up an FBS with a fixed high signal
strength value in the configuration. In contrast, to incorporate more
sophisticated attackers, we clone legitimate base station IDs and different
parameters, change signatures and signal strength, and turn them on and off to
evade detection. We introduce mobility in the dataset by using UEs with the
mobile endpoints in POWDER that are mounted in the campus shuttles. Benign
handovers are initiated when a shuttle with a mobile endpoint moves from the
vicinity of one base station to another, and the signal quality received at
the UE changes. To initiate a fake handover through FBSes, we spawn an FBS
near a UE with high signal strength and make the UE initiate an unsuccessful
handover. We combine all these scenarios in our experiments, listed in Table
9.

Implementation of handover capability in the srsUE.  The UEs usually initiate
handovers by sending a TrackingAreaUpdateRequest message. This message is not
implemented in the current release of srsRAN. We implement the
TrackingAreaUpdateRequest message in srsUE and make it able to initiate a
handover request by sending a TrackingAreaUpdateRequest message.

Implementation and Execution of MSAs.  We implement all the attacks listed in
Table 1 in srsRAN and execute them in the experimental setup we created in
POWDER. For a less sophisticated attacker, we implement the MSAs in srsRAN as
their definition, but for a more sophisticated attacker, we reshape the
packets in srsRAN and change the attack patterns by changing the packet
sending orders. All the traces from different components are captured for both
NAS and RRC layer.

Data Processing: We decode the packets using tshark (tshark, ). and use a
Python script for protocol filtering, packet field extraction, packet
alignment, and handling missing values. Lastly, we use scikit-learn’s
LabelEncoder to encode the categorical fields into numerical representations.

Data Labeling: We store the predefined attack labels in a dictionary and
assign each packet a label according to their reason for generation. The NAS
layer packets (fewer in number) are labeled with a script that checks each
packet and assigns a label according to the dictionary. The RRC layer packets
(which are much more numerous) are labeled in batches of intervals by
detecting attack intervals in the NAS layer packet sequences. There are only
two labels (benign and malicious) for the FBS dataset, whereas the MSA dataset
has multiple labels, one for each type of attack and one for benign packets.
Like before, we design a script that labels the malicious packets with a
different MSA-specific label for the attack it was generated from.

###  5.2. FBS Detection

Train-Test Split. We use a Python script to split our experimental dataset for
training and testing that does roughly $80-20$ split preserving the sequence
of packets. For example, if we have $10$ instances of the FBS experiment, we
append $8$ experiment’s data to the train set and $2$ to the test set. This
way we preserve the sequence of packets while ensuring the standard split,
also no experimental trace is cut in between.

Training Models. For training the Sequential-LSTM, we use a custom training
pipeline in Python that takes sequence length as a hyperparameter. All the
libraries used to train the ML models are listed in Table 2.

###  5.3. MSA Recognition

For NAS layer packets, we create a node for every unique value of the
nas_eps_nas_msg_emm_type_value field, the packet name for NAS layer packets.
Similarly, for the RRC layer packets, we create a node for every unique value
of the lte-rrc_c1_showname field, which is the packet name for the RRC layer
packets. Every packet maps to a node in the graph corresponding to its packet
name. We add an edge from the node representing one packet to the node
representing the next packet in the sequence and label that edge with the same
label we labeled the next packet. This denotes if the transition was benign or
due to an attack. The containers and libraries to create the directed graphs
and graph learning are summarized in Table 2.

###  5.4. Deployment and Integration

To deploy $\mathsf{FBSDetector}$, we use Mobileinsight (Mobileinsight, ) to
parse the baseband traces in the mobile phones, TensorflowLite (tflite, ) to
run the ML models, and Flutter (flutter, ) to build the app.

##  6\. Evaluation

We evaluate the effectiveness of $\mathsf{FBSDetector}$ based on the following
research questions:

:

RQ1. What is the distribution of the dataset, and what key features can be
extracted from the dataset?

:

RQ2. Why a sequential model is essential for FBS detection? How does the
weighted confidence-based fusion method for combining predictions of NAS and
RRC packets further improve performance? How does graph learning improve MSA
recognition performance?

:

RQ3. What is the memory and power consumption of the detection pipeline? How
much time does the inference take?

:

RQ4. Was $\mathsf{FBSDetector}$ deployed and tested in a real-world setup? How
does it contrast with existing FBS detection solutions?

:

RQ5. Is $\mathsf{FBSDetector}$ robust and generalizable against reshaping and
unseen attacks?

In what follows, we delve into the details and answers to those research
questions.

###  6.1. RQ1. Characteristics, Distribution, Feature, and Patterns of the
Dataset

To identify the characteristics and gain insights about the dataset, we show
different distributions of the packets in our dataset.

Total number of packets. There are $\mathsf{1662}$ NAS layer packets in the
FBS dataset in total, $\mathsf{907}$ benign and $\mathsf{755}$ malicious, and
there are $\mathsf{339813}$ RRC layer packets in total, $\mathsf{338195}$
benign and $\mathsf{1618}$ malicious. For the MSA dataset, there are
$\mathsf{16804}$ NAS layer packets, $\mathsf{502}$ benign and $\mathsf{15962}$
malicious, and there are $\mathsf{393684}$ RRC layer packets,
$\mathsf{338195}$ benign and $\mathsf{1618}$ malicious. The detailed
distributions are shown in Table 10 and 15 in appendix section A. From the
distributions, we can see that the number of packets during different
experiments is consistent, that is, the number of benign and malicious packets
is in a certain proportion in both NAS and RRC layers and they do not deviate
much from this proportion. This shows the consistency in our dataset.

Distribution of different NAS and RRC layer packets. We can see the
distribution of NAS and RRC layer packets in the combined FBS detection
dataset in Table 11 and 12 respectively in the Appendix Section A. Figure 6(a)
and 6(b) show this distribution graphically. The same observation can be made
from the distribution of MSA packets in Table 13 and 14 and Figure 6(c) and
6(d).

FBS sequence length. The distribution of FBS sequence length in the NAS and
RRC layer is shown in Figure 7(a) and 7(b). The figures show that for NAS
layer packets, most sequences are between $8$ and $20$. For RRC layer packets,
it is between $80$ and $120$. This is a significant observation because
sequence length is a hyperparameter of our model, and its performance depends
mainly on tuning this hyperparameter properly. We discuss the impact of
sequence length on model performance in our answer to RQ2.

(a) NAS packets in FBS

(b) RRC packets in FBS

(c) NAS packets in MSAs

(d) RRC packets in MSAs

Figure 6. Distribution of NAS and RRC packets in $\mathsf{FBSAD}$ and
$\mathsf{MSAD}$

###  6.2. RQ2. Sequential Training, Graph Learning, Predictions Combining

Sequential Training for a More Accurate Detection of FSBes. Like legitimate
base stations, FBSes generate sequences of messages. Figures 7(a) and 7(b)
show the distribution of the length of the sequences. As mentioned in Section
4, to take into account the context for each packet, we use a Sequential-LSTM
model taking the sequence length as a hyperparameter. The experimental results
reported in Table 3 demonstrate the superior performance of this approach over
other models. Though Sequential-LSTM outperforms its closest competitor
XGBoost, in detecting NAS layer packets with a small improvement in F1-Score
and accuracy (2%), the most substantial enhancement is observed with a 6%
increase in recall. Improving recall in malicious traffic classification means
the model is better at capturing a larger proportion of actual threats,
reducing the chances of missing malicious activity. This is crucial for
minimizing false negatives and enhancing overall detection effectiveness.
Figures 7(c) and 7(d) show the impact of sequence length on the detection
performance. Models perform better when the sequence length is between $9-15$
for NAS layer packets and $80-120$ for RRC layer packets. This is because, in
an FBS session, based on the communication process of the FBSes, packets
exchanged between the FBSes and the UEs in the NAS and RRC layer follow a
specific pattern. These patterns are captured better when the sequence lengths
are set in the range that can accommodate the patterns completely, and models
trained on these chunks of packets that contain these patterns can
sufficiently learn better about the attack.

Model | NAS | RRC
---|---|---
| Precision | Recall | F1-Score | Accuracy | Precision | Recall | F1-Score | Accuracy
Random Forest | 0.85 | 0.97 | 0.82 | 0.84 | 0.68 | 0.81 | 0.16 | 0.69
SVM | 0.81 | 0.21 | 0.70 | 0.59 | 0.66 | 0.79 | 0.00 | 0.66
Decision Tree | 0.86 | 0.90 | 0.81 | 0.82 | 0.75 | 0.84 | 0.53 | 0.76
XGBoost | 0.89 | 0.91 | 0.84 | 0.84 | 0.93 | 0.87 | 0.79 | 0.84
k-NN | 0.86 | 0.81 | 0.80 | 0.79 | 0.86 | 0.89 | 0.82 | 0.87
Naïve Bayes | 0.53 | 0.97 | 0.35 | 0.58 | 0.90 | 0.08 | 0.52 | 0.37
Logistic Regression | 0.50 | 0.68 | 0.69 | 0.53 | 0.74 | 0.85 | 0.50 | 0.71
CNN | 0.86 | 0.39 | 0.57 | 0.52 | 0.94 | 0.67 | 0.78 | 0.66
FNN | 0.79 | 0.85 | 0.68 | 0.73 | 0.80 | 0.88 | 0.84 | 0.78
LSTM | 0.86 | 0.65 | 0.72 | 0.71 | 0.89 | 0.56 | 0.71 | 0.59
Sequential-LSTM | 0.89 | 0.97 | 0.86 | 0.86 | 0.94 | 0.97 | 0.95 | 0.91
Table 3. Performance of ML models in FBS detection from NAS and RRC layer packets Model | Precision | Recall | F1-Score | Accuracy
---|---|---|---|---
Sequential-LSTM (NAS) | 0.89 | 0.97 | 0.86 | 0.86
Sequential-LSTM (RRC) | 0.94 | 0.97 | 0.95 | 0.91
Combined Prediction | 0.96 | 0.98 | 0.97 | 0.92
Table 4. Performance of FBS detection after combining predictions

Graph Learning in MSA Recognition. MSAs follow specific patterns, as shown in
Figure 5. We train graph learning models to learn these patterns, and Table 5
shows that learning the patterns, the graph learning models perform better
than any other models in recognizing MSAs, GraphSAGE performs the best among
all. So we use GraphSAGE as our MSA recognition model. Though the improvement
seen in accuracy might seem small (1%), a significant improvement is seen in
precision (4-8%), recall (10-12%) and f1-score (8-10%). It is worth noting
that Sequential-LSTM also achieves competitive performance compared to the
other ML models.

Model | NAS | RRC
---|---|---
Precision | Recall | F1-Score | Accuracy | Precision | Recall | F1-Score | Accuracy
Random Forest | 0.617 | 0.628 | 0.454 | 0.76 | 0.343 | 0.278 | 0.254 | 0.42
SVM | 0.088 | 0.200 | 0.122 | 0.44 | 0.076 | 0.200 | 0.110 | 0.38
Decision Tree | 0.342 | 0.402 | 0.356 | 0.66 | 0.076 | 0.200 | 0.110 | 0.38
XGBoost | 0.794 | 0.573 | 0.786 | 0.84 | 0.586 | 0.530 | 0.548 | 0.47
k-NN | 0.734 | 0.702 | 0.706 | 0.81 | 0.484 | 0.454 | 0.392 | 0.47
Naive Bayes | 0.440 | 0.384 | 0.274 | 0.29 | 0.440 | 0.258 | 0.178 | 0.11
Logistic Regression | 0.088 | 0.200 | 0.122 | 0.44 | 0.412 | 0.318 | 0.300 | 0.49
CNN | 0.132 | 0.274 | 0.114 | 0.22 | 0.072 | 0.259 | 0.106 | 0.36
FNN | 0.128 | 0.282 | 0.222 | 0.24 | 0.274 | 0.220 | 0.144 | 0.40
LSTM | 0.150 | 0.320 | 0.200 | 0.49 | 0.128 | 0.164 | 0.130 | 0.30
Sequential-LSTM | 0.744 | 0.726 | 0.784 | 0.85 | 0.408 | 0.342 | 0.348 | 0.53
Graph Attention Network | 0.868 | 0.672 | 0.865 | 0.84 | 0.316 | 0.338 | 0.298 | 0.36
Graph Attention Network v2 | 0.842 | 0.864 | 0.857 | 0.83 | 0.504 | 0.411 | 0.421 | 0.42
Graph Convolutional Network | 0.832 | 0.754 | 0.818 | 0.80 | 0.306 | 0.464 | 0.352 | 0.40
Graph Transformer | 0.836 | 0.882 | 0.841 | 0.81 | 0.436 | 0.495 | 0.430 | 0.41
GraphSAGE | 0.872 | 0.924 | 0.883 | 0.85 | 0.633 | 0.561 | 0.557 | 0.59
Table 5. Performance of MSA recognition from NAS and RRC layer packets Model | Precision | Recall | F1-Score | Accuracy
---|---|---|---|---
GraphSAGE (NAS) | 0.867 | 0.879 | 0.856 | 0.850
GraphSAGE (RRC) | 0.611 | 0.480 | 0.52 | 0.590
Combined Prediction | 0.875 | 0.901 | 0.874 | 0.865
Table 6. Performance of MSA recognition after combining predictions

Weighted Confidence-based Fusion Method for Combining Predictions. The
performance after combining the predictions of NAS and RRC packets using the
weighted confidence-based fusion method for FBS detection and MSA recognition
are shown in Table 4 and Table 6, respectively. Leveraging knowledge from both
layers, this method improves all the performance metrics from the best model
among the two and makes the system robust, especially by improving recall.

(a) NAS sequence length

(b) RRC sequence length

(c) Accuracy in NAS

(d) Accuracy in RRC

Figure 7. Distribution and impact of NAS and RRC sequence length in FBS
detection

Experimental Setup. We provide detailed information about the experimental
setup, including the training hardware specifications and model
hyperparameters, in Appendix Section B.

###  6.3. RQ3. Overhead Analysis: Time Consumption, Memory Consumption, Power
Consumption

We evaluate the overhead of $\mathsf{FBSDetector}$ based on several criteria.
Figure 8(a) shows the time required to predict packets, which increases
linearly with the number of packets. The slope of the increase is minimal,
which means that the solution can scale to high throughput applications.
Figure 8(c) shows the memory consumption and Figure 8(b) shows the power
consumption of $\mathsf{FBSDetector}$, consumed in packet processing and
running ML model on the processed packets to generate inference. The trend of
power consumption is also linearly increasing, with a small slope and the
trend of memory consumption is decreasing, which can result from multiple
hardware-level optimizations by the operating system. Compared to a recent
approach (PHOENIX, ), which uses an average of 4 mW power,
$\mathsf{FBSDetector}$ uses less than 2 mW of power to detect an FBS. These
results show that $\mathsf{FBSDetector}$ is a promising solution for
deployment in real-world systems, having negligible overhead.

(a) Time consumption overhead

(b) Power consumption overhead

(c) Memory consumption overhead

Figure 8. Performance overhead of $\mathsf{FBSDetector}$

###  6.4. RQ4. Real-World Validation and Comparison with Existing Solutions

Real-world validation. To validate $\mathsf{FBSDetector}$’s performance
against threats in the wild in real environments, we create a testbed in our
controlled lab environment. We used (1) One USRP B210 (USRP-B210, ), (2) One
engineering laptop and (3) a Google Nexus 5 smartphone with a Google Fi sim
card that has the $\mathsf{FBSDetector}$ app installed. We used Open5GS
(Open5GS, ) for the core network and srsRAN (srsRAN, ) for the base station.
Following the standard approaches, we create and spawn an FBS using the laptop
as the core network and the USRP B210 SDR as the base station. We place the
FBS near the smartphone with the $\mathsf{FBSDetector}$ app, run experiments
with different scenarios for FBS detection and MSA recognition and measure
detection performance. We test with (1) varying attacker capabilities (from
less sophisticated to more sophisticated), (2) mobility (limited within the
lab environment), (3) benign handovers and FBS handover attempts, (4) unseen
attacks, and (5) reshaped attacks. The results are analogous to the similar
experiments we conducted in POWDER.

Comparison with existing solutions. We compare $\mathsf{FBSDetector}$ with
different real-world solutions. The criteria for the comparison are: (1) The
solution must not require any changes in the protocol. (2) The source of the
solution must be available and maintained. (3) No additional hardware is
required for operation. The comparison summary in Table 7 shows that only
Android IMSI Catcher (AIMSICD) (AIMSICD, ) and SnoopSnitch (SnoopSnitch, )
satisfy all comparison criteria. So we test them in our testbed by creating an
FBS in our controlled lab environment and spawning it near a mobile device
with AIMSICD and SnoopSnitch installed and running. We ran the experiment
several times, but neither of the solutions could detect the FBS, whereas
$\mathsf{FBSDetector}$ detected the FBS every time.

Solution |  Supports 4G |  No Change in Protocol |  No Additional Hardware Required |  Source Available |  Detects FBS
---|---|---|---|---|---
Crocodile Hunter(Cooper2021, ) |  ✓ |  ✓ |  ✗ |  ✗ |  -
Darshak(Darshak, ) |  ✗ |  - |  - |  - |  -
Baron(BARON, ) |  ✓ |  ✗ |  - |  - |  -
Phoenix(PHOENIX, ) |  ✓ |  ✓ |  ✓ |  ✗ |  -
Android IMSI Catcher(AIMSICD, ) |  ✓ |  ✓ |  ✓ |  ✓ |  ✗
SnoopSnitch(SnoopSnitch, ) |  ✓ |  ✓ |  ✓ |  ✓ |  ✗
$\mathsf{FBSDetector}$ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓
Table 7. Comparison of $\mathsf{FBSDetector}$ with existing solutions

###  6.5. RQ5. Detecting Unseen and Reshaped Attacks: Robustness and
Generalizability

To establish a benchmark and prove the robustness and generalizability of
$\mathsf{FBSDetector}$, we evaluate its capability to detect unseen and
reshaped attacks.

Unseen Attacks: $\mathsf{FBSDetector}$ can detect unseen attacks by leveraging
anomalies in behavior that deviate from benign patterns. In cases where it
encounters an attack it has not seen before, there will be a misclassification
into an existing class. However, the attack would not go undetected, as the
deviation from established benign behavior will still trigger an alert,
ensuring that all anomalies are identified and addressed. We test this by
keeping the stealthy kickoff attack data unseen during training. Table 8 shows
that most of the unseen attack packets ($86.15$%) are classified as paging
channel hijacking attacks, closest to the stealthy kickoff attack. This proves
the capability of $\mathsf{FBSDetector}$ to detect unseen attacks and to
generalize.

Classified as | Unseen Attack Detection | Reshaped Attack Detection
---|---|---
Benign |  Stealthy Kickoff | Total | Benign |  Reshaped Paging Channel Hijacking | Total
True labels | 133 |  65 | 198 | 145 |  95 | 240
Benign | 128 |  3 | 131 | 136 |  5 | 141
Paging Channel Hijacking | 3 |  56 | 59 | 6 |  82 | 88
Numb Attack | 2 |  6 | 8 | 2 |  6 | 8
Panic Attack |  |  |  | 1 |  2 | 3
Table 8. Unseen and reshaped attack detection

Reshaped Attacks: In case of attack reshaping, especially for attackers with
more sophisticated capabilities, even if the attacker is aware of the presence
of $\mathsf{FBSDetector}$ and reshapes the MSA completely to evade detection,
$\mathsf{FBSDetector}$ can still detect it. This reshaped behavior deviates
from benign behavior and overlaps with the original attack that was reshaped,
and $\mathsf{FBSDetector}$ is able to detect it from this deviation and
overlap. To test $\mathsf{FBSDetector}$’s capability to detect reshaped
attacks properly, we reshape the paging channel hijacking attack by changing
different fields in the paging messages and changing the paging intervals.
Table 8 shows that most of the reshaped attack packets ($86.32$%) are
classified as paging channel hijacking attacks, which is the original attack.
This proves the capability of $\mathsf{FBSDetector}$ to detect reshaped
attacks.

##  7\. Related Work

Several approaches have been proposed to address the challenge of detecting
FBSes. We now present an overview of those approaches and highlight their key
contributions and limitations.

Fake base station detection and defense. The approaches proposed in
(OVERWATCH, ; SeaGlass, ) require installing expensive hardware and in most
cases they are proprietary. The techniques in (Cooper2021, ; Darshak, ;
AIMSICD, ; SnoopSnitch, ; PHOENIX, ) are based on several simple heuristics
and signatures which often entail high false positive rates. The approach in
(Li2017FBSRadarUF, ) depends on crowd-sourced data, which is not a practical
solution for scaling up. ML based efforts are effective in detecting attacks
and anomalies from traces (Trustin5G, ; Onetraceisallittakes, ; NakarmiML2022,
). However, they work on small datasets generated in simulated environments
that lack diversity; also because they work only in the data plane, they
cannot detect MSAs. Previous works have shown that information related to the
connection between a UE and a base station can be used to reason about the
authenticity of the base station (Arslan2019, ; Hamad2019ICCAIS, ;
Dabrowski2016TheMS, ; CatchMeIfYouCan, ; Huang2018IdentifyingTF, ;
Li2017FBSRadarUF, ; Murat, ; Karaçay2021, ). However, these solutions require
changes to the protocol and additional hardware. Recent efforts have
introduced certificate-based solutions and digital signatures
(Singla2020ProtectingT4, ; Look-Before-You-Leap, ; FBSSpecifications, ;
Trustin5G, ) for base station authentication. In (BARON, ; Insecure-
Connection-Root-of-All-Evil, ; Look-Before-You-Leap, ), the authors propose a
defense methodology to enable UEs to determine whether a base station is
legitimate or fake by efficient broadcast authentication protocol. These
techniques introduce additional authentication methods requiring critical
specifications modifications and add overhead.

Cellular network security. Over the years, several research efforts have
focused on the security of cellular networks. DoLTEst (DoLTEst, ) introduced
an in-depth downlink negative testing framework for 4G devices. LTEFuzz
(LTEFuzz, ) extensively investigates the security aspects of control plane
procedures in 4G networks. Methods for stealthy tracking and eavesdropping are
introduced in (kotuliak2022ltrack, ; hoang2023ltesniffer, ). Authors in
(erni2022adaptover, ; yang2019hiding, ) demonstrate adaptive physical
overshadowing attacks in cellular networks. REVOLTE (CallMeMaybe, ) exploits
an LTE implementation flaw to recover the contents of an encrypted VoLTE call
and discuss short and long-term countermeasures. A comprehensive layer two
security analysis is conducted by (ALTER, ) and a proof-of-concept 5G SUCI-
Catcher attack is demonstrated by (chlosta20215g, ). The unmitigated handover
vulnerabilities and potential countermeasures are studied by (handover, ).
Klischies et al. (InstructionsUnclear, ) systematically discover undefined
behavior in cellular specifications. IMP4GT (rupprecht-20-imp4gt, ) analyzes
missing integrity protection of the user plane in mutual authentication
protocol. Through different types of documentation analysis, Chen et al.
(BookwormGame, ; Sherlock, ; chen2022seeingtheforest, ) discover
vulnerabilities in cellular network protocols automatically. The authors in
(altaf2016, ; altaf_wisec18, ; Borgaonkar2019NewPT, ; ross2023towards, ) study
the simultaneous practical attacks against privacy and availability in
multiple cellular networks. The capability to abuse 5g’s warning and emergency
systems is demonstrated by (bitsikas2022you, ). Karakoc et al. analyze
bidding-down attacks(BiddingDownAttacks, ). Frameworks for protocol analysis
(Hussain2018LTEInspectorAS, ; prochecker, ; 5GReasoner, ) and implementation
analysis (DIKEUE, ) aim to find inconsistencies in specifications and
implementations that can lead to critical vulnerabilities. The exploitation
and protection of the paging protocol are investigated in
(Hussain2019PrivacyAT, ; Singla2020ProtectingT4, ). Despite these efforts,
detecting FBSes remains a challenging problem due to the ineffectiveness and
impracticality of the existing solutions against the rapid evolution of attack
techniques.

##  8\. Discussion

Applicability to 5G. To the best of our knowledge, no open-source protocol
stack for the standalone 5G core network supports handover, which is a
prerequisite for experimenting with FBSes and conducting FBS-based MSAs.
Therefore, we leave the detection of FBSes and MSAs in 5G cellular networks
with $\mathsf{FBSDetector}$ as a future work. However we believe that, based
on 4G, our approach is equally applicable to 5G. For example, like 4G, 5G has
a multi-layer design with most of the procedures unchanged from 4G. Thus, the
multi-layer protocol filtering, distribution and sequence of packets, and
other insights will largely remain the same when porting
$\mathsf{FBSDetector}$ to 5G.

Ethical Considerations. Safe practices were always employed during the
experiments in the controlled lab environment. The experiments involving radio
frequencies were conducted inside a Faraday cage. Only authorized personnel
with the testing devices were allowed inside the lab. The authors used their
SIM cards for the experiments, and they did not face any problems using the
SIM cards afterward. To maintain ethical standards in the FBSDetector app
while capturing packets, it asks for consent from its user before each
operation. User data confidentiality is a top priority, with robust encryption
and storage measures in place. An opt-out option ensures user autonomy. The
app is committed to aligning with ethical principles, prioritizing user trust
and privacy.

Recent efforts by companies to defend FBS. In 2021, Google released an
optional feature for Android to turn off the ability to connect to 2G cell
sites(New-Ways-to-Defeat-Cell-Site-Simulators, ). Google announced another new
mobile security setting for Android which allows users to prevent their phone
from using a null cipher when making a connection with a cell
tower(googleblog202308, ). When the null cipher is used, communications are
sent in the clear and not encrypted. Apple has also taken steps to protect
users against cell site simulators and announced that in iOS 17, out September
18, iPhones will not connect to insecure 2G mobile towers if they are placed
in Lockdown Mode(New-Ways-to-Defeat-Cell-Site-Simulators, ). As the name
implies, Lockdown Mode is a setting that locks down several features for
people who are concerned about being attacked by mercenary spyware.

##  9\. Conclusion And Future Work

In this paper, we have presented $\mathsf{FBSDetector}$, an ML-based FBS and
MSA detection system from cellular network, which leverages network traces at
Layer 3. To train the ML models, we have created $\mathsf{FBSAD}$ and
$\mathsf{MSAD}$, the _first-ever_ large-scale real-world datasets of
$\mathsf{6.6}$ GB combined, containing $\mathsf{751963}$ packets. Our novel
models, designed specially to detect FBSes and MSAs, perform better than any
state-of-the-art model and detect FBSes with a high accuracy of
$\mathsf{92\%}$ and a low false positive rate of $\mathsf{5.96\%}$, and MSAs
with $\mathsf{86\%}$ accuracy and $\mathsf{7.82\%}$ false positive rate. We
deploy $\mathsf{FBSDetector}$ on an Android app that effectively detects FBSes
in all the tested real-world scenarios.

Future work: In the future, we will port $\mathsf{FBSDetector}$ to 5G and
focus on detecting FBSes within the emerging Open Radio Access Network (ORAN)
environment, employing advanced machine learning algorithms through xApps. We
will also incorporate federated learning in UE clusters at the network edge to
detect FBSes and MSAs.

## Acknowledgements

This work is supported by NSF Grant No. 2112471. We thank the POWDER team for
their support in utilizing the platform.

## References

  * [1] HOW MANY PEOPLE HAVE SMARTPHONES IN 2023?. https://www.oberlo.com/statistics/how-many-people-have-smartphones.
  * [2] Forecast number of mobile devices worldwide from 2020 to 2025 (in billions)*. https://www.statista.com/statistics/245501/multiple-mobile-device-ownership-worldwide/.
  * [3] Mitziu Echeverria, Zeeshan Ahmed, Bincheng Wang, M. Fareed Arif, Syed Rafiul Hussain, and Omar Chowdhury.  PHOENIX: device-centric cellular network protocol monitoring using runtime verification.  In 28th Annual Network and Distributed System Security Symposium, NDSS 2021, virtually, February 21-25, 2021. The Internet Society, 2021.
  * [4] Syed Rafiul Hussain, Mitziu Echeverria, Ankush Singla, Omar Chowdhury, and Elisa Bertino.  Insecure connection bootstrapping in cellular networks: The root of all evil.  In Proceedings of the 12th Conference on Security and Privacy in Wireless and Mobile Networks, WiSec ’19, page 1–11, New York, NY, USA, 2019. Association for Computing Machinery.
  * [5] CheolJun Park, Sangwook Bae, BeomSeok Oh, Jiho Lee, Eunkyu Lee, Insu Yun, and Yongdae Kim.  $\\{$DoLTEst$\\}$: In-depth downlink negative testing framework for $\\{$LTE$\\}$ devices.  In 31st USENIX Security Symposium (USENIX Security 22), pages 1325–1342, 2022.
  * [6] Hongil Kim, Jiho Lee, Eunkyu Lee, and Yongdae Kim.  Touching the untouchables: Dynamic security analysis of the lte control plane.  In 2019 IEEE Symposium on Security and Privacy (SP), pages 1153–1168, 2019.
  * [7] Syed Rafiul Hussain, Omar Chowdhury, Shagufta Mehnaz, and Elisa Bertino.  Lteinspector: A systematic approach for adversarial testing of 4g lte.  In Network and Distributed System Security Symposium, 2018.
  * [8] Altaf Shaik, Jean-Pierre Seifert, Ravishankar Borgaonkar, N. Asokan, and Valtteri Niemi.  Practical attacks against privacy and availability in 4g/lte mobile communication systems.  In 23rd Annual Network and Distributed System Security Symposium, NDSS 2016, San Diego, California, USA, February 21-24, 2016. The Internet Society, 2016.
  * [9] Syed Rafiul Hussain, Mitziu Echeverria, Imtiaz Karim, Omar Chowdhury, and Elisa Bertino.  5greasoner: A property-directed security and privacy analysis framework for 5g cellular network protocol.  In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS ’19, page 669–684, New York, NY, USA, 2019. Association for Computing Machinery.
  * [10] Syed Rafiul Hussain, Mitziu Echeverria, Omar Chowdhury, Ninghui Li, and Elisa Bertino.  Privacy attacks to the 4g and 5g cellular paging protocols using side channel information.  Proceedings 2019 Network and Distributed System Security Symposium, 2019.
  * [11] Simon Erni, Martin Kotuliak, Patrick Leu, Marc Roeschlin, and Srdjan Capkun.  Adaptover: adaptive overshadowing attacks in cellular networks.  In Proceedings of the 28th Annual International Conference on Mobile Computing And Networking, pages 743–755, 2022.
  * [12] DHS confirms it has detected evidence of mobile snooping devices around DC. https://www.cnn.com/2018/04/03/politics/dhs-stingrays-washington-dc/index.html.
  * [13] Gang Of Drivers Caught Using Stingrays To Send Fake Links And Steal Cash. https://thainewsroom.com/2023/05/25/gang-of-drivers-caught-using-stingrays.
  * [14] Merlin Chlosta, David Rupprecht, Christina Pöpper, and Thorsten Holz.  5g suci-catchers: Still catching them all?  In Proceedings of the 14th ACM Conference on Security and Privacy in Wireless and Mobile Networks, pages 359–364, 2021.
  * [15] Ankush Singla, Syed Rafiul Hussain, Omar Chowdhury, Elisa Bertino, and Ninghui Li.  Protecting the 4g and 5g cellular paging protocols against security and privacy attacks.  Proceedings on Privacy Enhancing Technologies, 2020:126 – 142, 2020.
  * [16] Ankush Singla, Rouzbeh Behnia, Syed Rafiul Hussain, Attila Yavuz, and Elisa Bertino.  Look before you leap: Secure connection bootstrapping for 5g networks to defend against fake base-stations.  In Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security, ASIA CCS ’21, page 501–515, New York, NY, USA, 2021. Association for Computing Machinery.
  * [17] 3GPP TS 33.809 Study on 5G security enhancements against False Base Stations (FBS): Certificate based solution for Protecting System Information Messages with Digital Signature in an NPN. https://www.3gpp.org/ftp/TSG_SA/WG3_Security/TSGS3_100Bis-e/Docs/S3-202717.zip.
  * [18] Gabriel K. Gegenhuber, Wilfried Mayer, Edgar R. Weippl, and Adrian Dabrowski.  Mobileatlas: Geographically decoupled measurements in cellular networks for security and privacy research.  In Joseph A. Calandrino and Carmela Troncoso, editors, 32nd USENIX Security Symposium, USENIX Security 2023, Anaheim, CA, USA, August 9-11, 2023. USENIX Association, 2023.
  * [19] Guillem Reus-Muns, Dheryta Jaisinghani, Kunal Sankhe, and Kaushik R. Chowdhury.  Trust in 5g open rans through machine learning: Rf fingerprinting on the powder pawr platform.  In GLOBECOM 2020 - 2020 IEEE Global Communications Conference, pages 1–6, 2020.
  * [20] Leyli Karaçay, Zeki Bilgin, Ayşe Bilge Gündüz, Pinar Çomak, Emrah Tomur, Elif Ustundag Soykan, Utku Gülen, and Ferhat Karakoç.  A network-based positioning method to locate false base stations.  IEEE Access, 9:111368–111382, 2021.
  * [21] Prajwol Kumar Nakarmi, Mehmet Akif Ersoy, Elif Ustundag Soykan, and Karl Norrman.  Murat: Multi-rat false base station detector.  CoRR, abs/2102.08780, 2021.
  * [22] Adrian Dabrowski, Nicola Pianta, Thomas Klepp, Martin Mulazzani, and Edgar Weippl.  Imsi-catch me if you can: Imsi-catcher-catchers.  In Proceedings of the 30th Annual Computer Security Applications Conference, ACSAC ’14, page 246–255, New York, NY, USA, 2014. Association for Computing Machinery.
  * [23] Cooper Quintin.  Detecting fake 4g LTE base stations in real time.  USENIX Association, February 2021.
  * [24] Arslan Ali and Georg Fischer.  Enabling fake base station detection through sample-based higher order noise statistics.  In 2019 42nd International Conference on Telecommunications and Signal Processing (TSP), pages 695–700, 2019.
  * [25] Hamad Alrashede and Riaz Ahmed Shaikh.  Imsi catcher detection method for cellular networks.  In 2019 2nd International Conference on Computer Applications & Information Security (ICCAIS), pages 1–6, 2019.
  * [26] Ke-Wen Huang and Huiming Wang.  Identifying the fake base station: A location based approach.  IEEE Communications Letters, 22:1604–1607, 2018.
  * [27] Leo Weissbart, Stjepan Picek, and Lejla Batina.  One trace is all it takes: Machine learning-based side-channel attack on eddsa.  Cryptology ePrint Archive, Paper 2019/358, 2019.  https://eprint.iacr.org/2019/358.
  * [28] Prajwol Kumar Nakarmi, Jakob Sternby, and Ikram Ullah.  Applying machine learning on rsrp-based features for false base station detection.  In Proceedings of the 17th International Conference on Availability, Reliability and Security, ARES ’22, New York, NY, USA, 2022. Association for Computing Machinery.
  * [29] Zhenhua Li, Weiwei Wang, Christo Wilson, Jian Chen, Chen Qian, Taeho Jung, Lan Zhang, Kebin Liu, Xiangyang Li, and Yunhao Liu.  Fbs-radar: Uncovering fake base stations at scale in the wild.  Proceedings 2017 Network and Distributed System Security Symposium, 2017.
  * [30] Darshak. https://github.com/darshakframework/darshak.
  * [31] The Android-IMSI-Catcher-Detector (short: AIMSICD). http://www.tea-after-twelve.com/about-us/our-authors/aimsicd.
  * [32] Alessandro Lotto, Vaibhav Singh, Bhaskar Ramasubramanian, Alessandro Brighente, Mauro Conti, and Radha Poovendran.  Baron: Base-station authentication through core network for mobility management in 5g networks.  In Proceedings of the 16th ACM Conference on Security and Privacy in Wireless and Mobile Networks, WiSec ’23, page 133–144, New York, NY, USA, 2023. Association for Computing Machinery.
  * [33] SnoopSnitch. https://play.google.com/store/apps/details?id=de.srlabs.snoopsnitch&hl=en_US&gl=US.
  * [34] OVERWATCH IMSI CATCHER DETECTION SERVICES. https://comsecllc.com/comsec-llc-adds-overwatch.
  * [35] SeaGlass: City-Wide IMSI-Catcher Detection. .https://news.ycombinator.com/item?id=27173717.
  * [36] Zhaowei Tan, Jinghao Zhao, Boyan Ding, and Songwu Lu.  CellDAM: User-Space, rootless detection and mitigation for 5g data plane.  In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23), pages 1601–1619, Boston, MA, April 2023. USENIX Association.
  * [37] Junqing Zhang, Guanxiong Shen, Walid Saad, and Kaushik Chowdhury.  Radio frequency fingerprint identification for device authentication in the internet of things.  IEEE Communications Magazine, pages 1–7, 2023.
  * [38] Apple and Google Are Introducing New Ways to Defeat Cell Site Simulators, But Is it Enough?. https://www.eff.org/deeplinks/2023/09/apple-and-google-are-introducing-new-ways-defeat-cell-site-simulators-it-enough.
  * [39] Non-Access-Stratum (NAS) protocol for Evolved Packet System (EPS); Stage 3. https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=1072.
  * [40] Evolved Universal Terrestrial Radio Access (E-UTRA); Radio Resource Control (RRC); Protocol specification. https://portal.3gpp.org/desktopmodules/Specifications/SpecificationDetails.aspx?specificationId=2440.
  * [41] Joe Breen, Andrew Buffmire, Jonathon Duerig, Kevin Dutt, Eric Eide, Mike Hibler, David Johnson, Sneha Kumar Kasera, Earl Lewis, Dustin Maas, Alex Orange, Neal Patwari, Daniel Reading, Robert Ricci, David Schurig, Leigh B. Stoller, Jacobus Van der Merwe, Kirk Webb, and Gary Wong.  Powder: Platform for open wireless data-driven experimental research.  In Proceedings of the 14th International Workshop on Wireless Network Testbeds, Experimental Evaluation and Characterization, WiNTECH’20, page 17–24, New York, NY, USA, 2020. Association for Computing Machinery.
  * [42] Sergei Chuprov, Leon Reznik, Antoun Obied, and Srujan Shetty.  How degrading network conditions influence machine learning end systems performance?  In IEEE INFOCOM 2022 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pages 1–6, 2022.
  * [43] David Johnson, Dustin Maas, and Jacobus Van Der Merwe.  Nexran: Closed-loop ran slicing in powder -a top-to-bottom open-source open-ran use case.  In Proceedings of the 15th ACM Workshop on Wireless Network Testbeds, Experimental Evaluation & CHaracterization, WiNTECH ’21, page 17–23, New York, NY, USA, 2021. Association for Computing Machinery.
  * [44] Jose Monterroso, Jacobus Van der Merwe, Kirk Webb, and Gary Wong.  Towards using the powder platform for rf propagation validation.  In IEEE INFOCOM 2021 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pages 1–6, 2021.
  * [45] Kirk Webb, Sneha Kumar Kasera, Neal Patwari, and Jacobus Van der Merwe.  Wimatch: Wireless resource matchmaking.  In IEEE INFOCOM 2021 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pages 1–6, 2021.
  * [46] Zhenghao Zhang.  Zcnet: Achieving high capacity in low power wide area networks.  In 2020 IEEE 17th International Conference on Mobile Ad Hoc and Sensor Systems (MASS), pages 702–710, 2020.
  * [47] Martin Kotuliak, Simon Erni, Patrick Leu, Marc Röschlin, and Srdjan Čapkun.  $\\{$LTrack$\\}$: Stealthy tracking of mobile phones in $\\{$LTE$\\}$.  In 31st USENIX Security Symposium (USENIX Security 22), pages 1291–1306, 2022.
  * [48] 5G - 4G-5G Subscribers March 2022 – Quarterly update. https://gsacom.com/paper/4g-5g-subscribers-march-2022-quarterly-update/.
  * [49] Mobile subscriptions outlook. https://www.ericsson.com/en/reports-and-papers/mobility-report/dataforecasts/mobile-subscriptions-outlook.
  * [50] Tracking the 4G Decade. https://blog.telegeography.com/tracking-the-4g-decade.
  * [51] The Mobile Economy 2023. https://www.gsma.com/mobileeconomy/wp-content/uploads/2023/03/270223-The-Mobile-Economy-2023.pdf.
  * [52] Number of LTE subscriptions worldwide from 2018 to 2023 (in billions)*. https://www.statista.com/statistics/206615.
  * [53] Qinli Zhang, Liangdong Qu, and Zhaowen Li.  Attribute reduction based on d-s evidence theory in a hybrid information system.  International Journal of Approximate Reasoning, 148:202–234, 2022.
  * [54] Yongchuan Tang, Xu Zhang, Ying Zhou, Yubo Huang, and Deyun Zhou.  A new correlation belief function in dempster-shafer evidence theory and its application in classification.  Scientific Reports, 13(1):7609, May 2023.
  * [55] Bedran Karakoc, Nils Fürste, David Rupprecht, and Katharina Kohls.  Never let me down again: Bidding-down attacks and mitigations in 5g and 4g.  2023\.
  * [56] Dempster–Shafer theory. https://en.wikipedia.org/wiki/Dempster%E2%80%93Shafer_theory.
  * [57] Open5GS. https://github.com/open5gs/open5gs.
  * [58] srsRAN. https://github.com/srsran/srsRAN.
  * [59] tshark. https://www.wireshark.org/docs/man-pages/tshark.html.
  * [60] Yuanjie Li, Chunyi Peng, Zengwen Yuan, Jiayao Li, Haotian Deng, and Tao Wang.  Mobileinsight: Extracting and analyzing cellular network information on smartphones.  In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking, MobiCom ’16, page 202–215, New York, NY, USA, 2016. Association for Computing Machinery.
  * [61] tensorflow-lite. https://www.tensorflow.org/lite/guide/inference?utm_campaign=Thoughts%20on%20HCI%20and%20Applied%20AI%20&utm_medium=email&utm_source=Revue%20newsletter.
  * [62] flutter. https://docs.flutter.dev/.
  * [63] USRP B210 SDR Kit - Dual Channel Transceiver (70 MHz - 6GHz) - Ettus Research.  https://www.ettus.com/all-products/ub210-kit/.
  * [64] Adrian Dabrowski, Georg Petzl, and Edgar R. Weippl.  The messenger shoots back: Network operator based imsi catcher detection.  In International Symposium on Recent Advances in Intrusion Detection, 2016.
  * [65] Tuan Dinh Hoang, CheolJun Park, Mincheol Son, Taekkyung Oh, Sangwook Bae, Junho Ahn, Beomseok Oh, and Yongdae Kim.  Ltesniffer: An open-source lte downlink/uplink eavesdropper.  2023\.
  * [66] Hojoon Yang, Sangwook Bae, Mincheol Son, Hongil Kim, Song Min Kim, and Yongdae Kim.  Hiding in plain signal: Physical signal overshadowing attack on $\\{$LTE$\\}$.  In 28th USENIX Security Symposium (USENIX Security 19), pages 55–72, 2019.
  * [67] David Rupprecht, Katharina Kohls, Thorsten Holz, and Christina Pöpper.  Call me maybe: Eavesdropping encrypted lte calls with revolte.  In Proceedings of the 29th USENIX Conference on Security Symposium, SEC’20, USA, 2020. USENIX Association.
  * [68] David Rupprecht, Katharina Kohls, Thorsten Holz, and Christina Pöpper.  Breaking lte on layer two.  In 2019 IEEE Symposium on Security and Privacy (SP), pages 1121–1136. IEEE, 2019.
  * [69] Evangelos Bitsikas and Christina Pöpper.  Don’t hand it over: Vulnerabilities in the handover procedure of cellular telecommunications.  In Annual Computer Security Applications Conference, ACSAC ’21, page 900–915, New York, NY, USA, 2021. Association for Computing Machinery.
  * [70] Daniel Klischies, Moritz Schloegel, Tobias Scharnowski, Mikhail Bogodukhov, David Rupprecht, and Veelasha Moonsamy.  Instructions unclear: Undefined behaviour in cellular network specifications.  In 32nd USENIX Security Symposium (USENIX Security 23), pages 3475–3492, Anaheim, CA, August 2023. USENIX Association.
  * [71] David Rupprecht, Katharina Kohls, Thorsten Holz, and Christina Pöpper.  Imp4gt: Impersonation attacks in 4g networks.  In ISOC Network and Distributed System Security Symposium (NDSS). ISOC, February 2020.
  * [72] Yi Chen, Yepeng Yao, XiaoFeng Wang, Dandan Xu, Chang Yue, Xiaozhong Liu, Kai Chen, Haixu Tang, and Baoxu Liu.  Bookworm game: Automatic discovery of lte vulnerabilities through documentation analysis.  In 2021 IEEE Symposium on Security and Privacy (SP), pages 1197–1214, 2021.
  * [73] Yi Chen, Di Tang, Yepeng Yao, Mingming Zha, XiaoFeng Wang, Xiaozhong Liu, Haixu Tang, and Baoxu Liu.  Sherlock on specs: Building LTE conformance tests through automated reasoning.  In 32nd USENIX Security Symposium (USENIX Security 23), pages 3529–3545, Anaheim, CA, August 2023. USENIX Association.
  * [74] Yi Chen, Di Tang, Yepeng Yao, Mingming Zha, XiaoFeng Wang, Xiaozhong Liu, Haixu Tang, and Dongfang Zhao.  Seeing the forest for the trees: Understanding security hazards in the $\\{$3GPP$\\}$ ecosystem through intelligent analysis on change requests.  In 31st USENIX Security Symposium (USENIX Security 22), pages 17–34, 2022.
  * [75] Altaf Shaik, Ravishankar Borgaonkar, Shinjo Park, and Jean-Pierre Seifert.  On the impact of rogue base stations in 4g/lte self organizing networks.  In Proceedings of the 11th ACM Conference on Security & Privacy in Wireless and Mobile Networks, WiSec ’18, page 75–86, New York, NY, USA, 2018. Association for Computing Machinery.
  * [76] Ravishankar Borgaonkar, Lucca Hirschi, Shinjo Park, and Altaf Shaik.  New privacy threat on 3g, 4g, and upcoming 5g aka protocols.  Proceedings on Privacy Enhancing Technologies, 2019:108 – 127, 2019.
  * [77] Alexander J Ross and Bradley Reaves.  Towards simultaneous attacks on multiple cellular networks.  In 2023 IEEE Security and Privacy Workshops (SPW), pages 394–405. IEEE, 2023.
  * [78] Evangelos Bitsikas and Christina Pöpper.  You have been warned: Abusing 5g’s warning and emergency systems.  In Proceedings of the 38th Annual Computer Security Applications Conference, pages 561–575, 2022.
  * [79] Imtiaz Karim, Syed Rafiul Hussain, and Elisa Bertino.  Prochecker: An automated security and privacy analysis framework for 4g lte protocol implementations.  In 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS), pages 773–785, 2021.
  * [80] Syed Rafiul Hussain, Imtiaz Karim, Abdullah Al Ishtiaq, Omar Chowdhury, and Elisa Bertino.  Noncompliance as deviant behavior: An automated black-box noncompliance checker for 4g lte cellular devices.  In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, CCS ’21, page 1082–1099, New York, NY, USA, 2021. Association for Computing Machinery.
  * [81] Android 14 introduces first-of-its-kind cellular connectivity security features. https://security.googleblog.com/2023/08/android-14-introduces-first-of-its-kind.html.

##  Appendix A Dataset Construction

###  A.1. FBS Dataset - $\mathsf{FBSAD}$

The description of different experiments for FBS data generation is given in
Table 9. The detailed distributions of the packets are shown in Table 10. The
distribution of NAS and RRC layer packets in $\mathsf{FBSAD}$ are shown in
Table 11 and 12 respectively.

Exp No. | Exp Description
---|---
Exp 1, 2 | Fake handover initiation by FBS
Exp 3, 10 | No fake base station, no handover
Exp 4 | No fake base station, handover tried
Exp 5-7, 17-19 | No fake base station, multiple benign handovers
Exp 8-9, 11-16 | Fake base station, fake handover attempted and recovered
Table 9. FBS Experiment Description Exp No. | Number of Packets
---|---
NAS | RRC
Benign | Malicious | Total | Benign | Malicious | Total
Exp1 | 33 | 43 | 76 | 2596 | 59 | 2655
Exp2 | 39 | 63 | 102 | 17099 | 25 | 17124
Exp3 | 49 | 0 | 49 | 10197 | 0 | 10197
Exp4 | 5 | 0 | 5 | 47 | 0 | 47
Exp5 | 61 | 0 | 61 | 3367 | 0 | 3367
Exp6 | 66 | 0 | 66 | 2189 | 0 | 2189
Exp7 | 73 | 0 | 73 | 4571 | 0 | 4571
Exp8 | 21 | 42 | 63 | 6088 | 137 | 6225
Exp9 | 13 | 81 | 94 | 5084 | 114 | 5198
Exp10 | 27 | 0 | 27 | 2696 | 0 | 2696
Exp11 | 74 | 101 | 175 | 9332 | 209 | 9541
Exp12 | 65 | 43 | 108 | 89874 | 133 | 90007
Exp13 | 73 | 198 | 271 | 12251 | 275 | 12526
Exp14 | 32 | 48 | 80 | 14569 | 327 | 14896
Exp15 | 62 | 94 | 156 | 10826 | 243 | 11069
Exp16 | 37 | 42 | 79 | 64568 | 96 | 64664
Exp17 | 128 | 0 | 128 | 57202 | 0 | 57202
Exp18 | 27 | 0 | 27 | 24605 | 0 | 24605
Exp19 | 22 | 0 | 22 | 1034 | 0 | 1034
| 907 | 755 | 1662 | 338195 | 1618 | 339813
Table 10. Number of packets of each layer in $\mathsf{FBSAD}$ Packet Name | Benign | Malicious | Total
---|---|---|---
Attach accept | 103 | 1 | 104
Attach complete | 103 | 1 | 104
Attach request | 105 | 173 | 278
Authentication failure | 1 | 172 | 173
Authentication reject | 3 | 0 | 3
Authentication request | 72 | 172 | 244
Authentication response | 71 | 0 | 71
Detach request | 17 | 1 | 18
EMM information | 103 | 1 | 104
Identity request | 44 | 98 | 142
Identity response | 44 | 98 | 142
Security mode command | 71 | 0 | 71
Security mode complete | 71 | 0 | 71
Service reject | 1 | 0 | 1
Tracking area update accept | 9 | 0 | 9
Tracking area update reject | 10 | 9 | 19
Tracking area update request | 60 | 26 | 86
| 888 | 752 | 1640
Table 11. NAS Layer Packet Distribution in $\mathsf{FBSAD}$ Packet Name | Benign | Malicious | Total
---|---|---|---
dlInformationTransfer | 95 | 51 | 146
ulInformationTransfer | 90 | 50 | 140
rrcConnectionRequest | 59 | 35 | 94
rrcConnectionSetup | 59 | 35 | 94
rrcConnectionSetupComplete | 59 | 35 | 94
rrcConnectionRelease | 55 | 36 | 91
systemInformationBlockType1 | 32 | 30 | 62
systemInformation | 32 | 30 | 62
rrcConnectionReconfiguration | 32 | 0 | 32
ueCapabilityInformation | 32 | 0 | 32
ueCapabilityEnquiry | 32 | 0 | 32
securityModeComplete | 32 | 0 | 32
securityModeCommand | 32 | 0 | 32
paging | 5 | 1 | 6
rrcConnectionReestablishmentRequest | 4 | 0 | 4
ueInformationRequest-r9 | 3 | 0 | 3
ueInformationResponse-r9 | 3 | 0 | 3
rrcConnectionReestablishmentReject | 2 | 0 | 2
rrcConnectionReestablishment | 1 | 0 | 1
| 659 | 303 | 962
Table 12. RRC Layer Packet Distribution in $\mathsf{FBSAD}$

###  A.2. MSA Dataset - $\mathsf{MSAD}$

The detailed distribution of the packets in $\mathsf{MSAD}$ is shown in Table
15. The distribution of NAS and RRC layer packets in $\mathsf{MSAD}$ are shown
in Table 13 and 14 respectively.

Packet Name |  Benign |  Energy Depletion Attack |  NAS Counter Desync Attack |  Numb Attack |  Paging Channel Hijacking Attack |  Total
---|---|---|---|---|---|---
Attach accept |  90 |  0 |  0 |  0 |  0 |  90
Attach complete |  90 |  0 |  0 |  0 |  0 |  90
Attach request |  84 |  108 |  0 |  128 |  28 |  348
Authentication reject |  0 |  0 |  0 |  8 |  0 |  8
Authentication request |  74 |  60 |  0 |  0 |  0 |  134
Authentication response |  74 |  60 |  0 |  0 |  0 |  134
Detach request |  27 |  0 |  0 |  0 |  0 |  27
EMM information |  90 |  0 |  0 |  0 |  0 |  90
Identity request |  48 |  24 |  0 |  0 |  8 |  80
Identity response |  48 |  24 |  0 |  0 |  8 |  80
Security mode command |  74 |  60 |  245 |  0 |  0 |  379
Security mode complete |  74 |  60 |  0 |  0 |  0 |  134
Security mode reject |  0 |  0 |  245 |  0 |  0 |  245
Service reject |  1 |  0 |  0 |  0 |  0 |  1
Tracking area update accept |  5 |  0 |  0 |  0 |  0 |  5
Tracking area update reject |  11 |  12 |  0 |  0 |  4 |  27
Tracking area update request |  43 |  12 |  1 |  8 |  4 |  68
|  833 |  420 |  491 |  144 |  52 |  1940
Table 13. NAS layer Packet Distribution in $\mathsf{MSAD}$ Packet Name |  Benign |  Energy Depletion Attack |  NAS Counter Desync Attack |  Numb Attack |  Paging Channel Hijacking Attack |  Total
---|---|---|---|---|---|---
dlInformationTransfer |  30 |  20 |  17 |  4 |  4 |  75
ulInformationTransfer |  30 |  24 |  11 |  0 |  5 |  70
systemInformationBlockType1 |  8 |  13 |  4 |  6 |  5 |  36
rrcConnectionRequest |  13 |  9 |  3 |  1 |  4 |  30
rrcConnectionSetup |  13 |  9 |  3 |  1 |  4 |  30
rrcConnectionSetupComplete |  12 |  9 |  3 |  1 |  4 |  29
rrcConnectionRelease |  8 |  9 |  3 |  0 |  4 |  24
systemInformation |  8 |  7 |  2 |  1 |  2 |  20
rrcConnectionReconfiguration |  10 |  0 |  0 |  0 |  0 |  10
ueCapabilityInformation |  9 |  0 |  0 |  0 |  0 |  9
securityModeCommand |  9 |  0 |  0 |  0 |  0 |  9
ueCapabilityEnquiry |  9 |  0 |  0 |  0 |  0 |  9
securityModeComplete |  9 |  0 |  0 |  0 |  0 |  9
ueInformationRequest-r9 |  2 |  0 |  0 |  0 |  0 |  2
ueInformationResponse-r9 |  2 |  0 |  0 |  0 |  0 |  2
paging |  1 |  0 |  0 |  0 |  0 |  1
|  173 |  100 |  46 |  14 |  32 |  365
Table 14. RRC Layer Packet Distribution in $\mathsf{MSAD}$ Attack Name | Exp No. | Number of Packets
---|---|---
NAS |  RRC
Detach Downgrade Attack | exp-1 |  49 |  1469
exp-2 |  22 |  830
exp-3 |  16 |  81
exp-4 |  16 |  78
exp-5 |  16 |  86
exp-6 |  21 |  124
Incarceration Attack | exp-1 |  66 |  171976
exp-2 |  24 |  109
exp-3 |  60 |  20081
exp-4 |  24 |  1169
exp-5 |  24 |  4079
exp-6 |  24 |  9067
Stealthy Kickoff Attack | exp-1 |  21 |  82
exp-2 |  33 |  6888
exp-3 |  25 |  90
exp-4 |  25 |  90
exp-5 |  25 |  91
exp-6 |  25 |  89
Panic Attack | exp-1 |  17 |  37
exp-2 |  31 |  1013
exp-3 |  33 |  87
exp-4 |  31 |  3847
exp-5 |  36 |  350
exp-6 |  51 |  402
NAS Counter Desync Attack | exp-1 |  1554 |  19783
exp-2 |  1554 |  148
exp-3 |  6159 |  1360
exp-4 |  1554 |  422
exp-5 |  1554 |  176
exp-6 |  3081 |  233
Energy Depletion Attack | exp-1 |  80 |  1731
exp-2 |  37 |  137
exp-3 |  29 |  589
exp-4 |  29 |  196
exp-5 |  29 |  109
exp-6 |  89 |  4457
exp-7 |  103 |  4519
exp-8 |  63 |  3130
exp-9 |  29 |  91
exp-10 |  105 |  76476
Paging Channel Hijacking Attack | exp-1 |  21 |  85
exp-2 |  21 |  83
exp-3 |  21 |  83
exp-4 |  21 |  86
exp-5 |  0 |  20
exp-6 |  21 |  83
Numb Attack | exp-1 |  24 |  108
exp-2 |  56 |  18399
exp-3 |  63 |  30087
exp-4 |  26 |  4478
exp-5 |  36 |  12011
exp-6 |  81 |  506
Table 15. Number of packets of each layer of different MSA experiment dataset

##  Appendix B Experimental Setup

###  B.1. Training Hardware.

For training the models, we utilized a Lenovo ThinkPad T480 equipped with 32GB
of memory and an Intel Core i7-8650U CPU @ 1.90GHz × 8. The operating system
was Ubuntu 22.04.1 LTS (64-bit), running GNOME Version 42.2.

###  B.2. Model Hyperparameters.

Sequential LSTM. The Sequential-LSTM model consists of an LSTM layer with $64$
units and a sigmoid activation function, configured not to return sequences.
The subsequent layer is a dense layer with a single unit and sigmoid
activation. For optimization, stochastic gradient descent (sgd) was chosen,
with a mean squared error (mse) loss function. The model’s performance is
assessed using accuracy as the main metric, complemented by the inclusion of a
custom metric, false positives, to evaluate its classification capabilities
further.

GraphSAGE. The graph model is implemented featuring a single SAGEConv layer.
The SAGEConv layer utilizes $2$ attention heads to capture graph-based
relationships. The model’s architecture is encapsulated within a PyTorch
Module, with the forward function defining the flow of information through the
single layer. Logarithmic softmax is employed for the final layer’s output
activation.

Other Graph Models. For the other graph models, we used the same
configurations as the GraphSAGE model with their own convolutional layer. For
example, for the Graph Attention Network, we used GATConv, and for the Graph
Convolutional Network, we used GCNConv.

Other Classification Models. The Random Forest Classifier and the Decision
Tree Classifier were configured with a Gini criterion, a maximum depth of 3.
The XGBoost Classifier, Support Vector Classifier (SVC), K-Nearest-Neighbors
(KNN) Classifier, Gaussian Naive Bayes and Logistic Regression adopted default
configurations.

Convolutional Neural Network (CNN). The CNN architecture comprises two Conv1D
layers with $32$ and $64$ filters, respectively, followed by a ReLU activation
function. MaxPooling1D layers with a pool size of $2$ were inserted after each
convolutional layer to downsample the spatial dimensions. A
GlobalAveragePooling1D layer was then employed to aggregate the spatial
information across the entire sequence. Subsequently, two Dense layers were
added with $64$ units and ReLU activation in the first, and a single unit with
a sigmoid activation in the final layer for classification. The model was
compiled using the Adam optimizer, binary cross-entropy loss function, and
accuracy as the metric for performance evaluation.

Feedforward Neural Network (FNN). The FNN architecture consists of three Dense
layers, with the first two layers containing $64$ units each and utilizing the
ReLU activation function. The final dense layer, with a single unit and a
sigmoid activation function, is employed for classification. The model was
compiled using the Adam optimizer, binary cross-entropy as the loss function,
and accuracy as the metric for assessing its performance.

##  Appendix C POWDER

POWDER (Platform for Open Wireless Data-driven Experimental Research) resolves
all these challenges and provides us with a way to generate high-quality,
real-world, over-the-air datasets of FBSes with all the real-world scenarios
involved, especially mobility. This invaluable resource enables us to develop
and test FBS detection algorithms in a realistic environment. POWDER is a
highly flexible city-scale scientific instrument that enables research at the
forefront of the wireless revolution [41] and is currently deployed in Salt
Lake City, Utah. POWDER is a partnership between the University of Utah, Salt
Lake City, and other public and private organizations (local, national, and
global).

###  C.1. Hardware and Mobility

The physical deployment of POWDER encompasses three key components: (1) Core
Component: This includes the Controlled RF Lab and the Metro Cloud situated
on-premise, connected to a remote cloud through a wide area network (WAN). (2)
Edge Component: This involves edge computing devices to facilitate processing
closer to data sources. (3) Radio Access Component: Comprising base stations,
fixed, and mobile endpoints, this component establishes the fundamental
wireless communication infrastructure. The physical deployment offers a
variety of configurable “coverage” scenarios, e.g., conventional macro-cell,
small-cell (enabled by the campus “dense” deployment), or combinations
thereof. Diversity in _mobility_ is provided by using mobile couriers that
have relatively predictable movement patterns (e.g., buses), less predictable
but bounded mobility (e.g., maintenance vehicles), and couriers that are
“controllable” (e.g., backpacks/portable endpoints that can be moved by
researchers that come on-site). Each deployed node consists of user-
programmable software-defined radios (SDRs), off-the-shelf (OTS) radio
equipment, RF front-ends, and antennas. An architectural overview of POWDER is
given in Figure 9.

###  C.2. Software and Profiles

In the context of POWDER, profiles refer to configurable sets of parameters
that define various aspects of the wireless network environment. These
parameters encompass hardware, network configurations, signal propagation
characteristics, mobility patterns, interference scenarios, and more. Profiles
make it possible to replicate real-world conditions for experimentation,
allowing one to create diverse scenarios, including those involving FBSes.
Using profiles, one can manipulate the network environment within POWDER’s
controlled setting and conduct experiments that closely mirror real-world
situations while avoiding legal and regulatory challenges. By creating
different profiles, one can effectively create real-world conditions. This
capability is a significant advantage, allowing one to conduct comprehensive
experiments and analyses within a controlled environment.

Figure 9. POWDER overview

POWDER feedback POWDER makes it possible to replicate real-world conditions
for experimentation. One can manipulate the network environment within
POWDER’s controlled setting and conduct experiments using POWDER profiles.
However, only a limited number of profiles with some fixed sets of parameters
are currently available at POWDER. Users can create their profiles according
to their requirements. Still, a new user might find it challenging to create a
profile because there is not enough documentation and tutorials on creating
these profiles. The incorporation of more profiles with diverse scenarios,
automation of the profile creation process, and extensive documentation and
tutorials will help users utilize POWDER’s capabilities to a great extent.

##  Appendix D Mobile App

Different components of the $\mathsf{FBSDetector}$ Android app are
demonstrated in Figure 10.

(a) Homepage of FBS Detector

(b) No Fake Base Station detected

(c) Fake Base Station detected

Figure 10. Android app for $\mathsf{FBSDetector}$