# Security for Emerging Miniaturized Wireless Biomedical Devices: Threat
Modeling with Application to Case Studies

Vladimir Vakhter1 Betul Soysal  Patrick Schaumont1 and Ulkuhan Guler1
1Department of Electrical and Computer Engineering, Worcester Polytechnic
Institute, Worcester, USA {vvvakhter; pschaumont<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

The landscape of miniaturized wireless biomedical devices (MWBDs) is rapidly
expanding as proactive mobile healthcare proliferates. MWBDs are diverse and
include various injectable, ingestible, implantable, and wearable devices.
While the growth of MWBDs increases the flexibility of medical services, the
adoption of these technologies brings privacy and security risks for their
users. MWBDs can operate with sensitive, private information and affect
patients through the use of stimulation and drug delivery. Therefore, these
devices require trust and need to be secure. Embedding protective mechanisms
into MWBDs is challenging because they are restricted in size, power budget,
as well as processing and storage capabilities. Nevertheless, MWBDs need to be
at least minimally securable in the face of evolving threats. The main intent
of this work is to make the primary stakeholders of MWBDs aware of associated
risks and to help the architects and the manufacturers of MWBDs protect their
emerging designs in a repeatable and structured manner. Making MWBDs securable
begins with performing threat modeling. This paper introduces a domain-
specific qualitative-quantitative threat model dedicated to MWBDs. The
proposed model is then applied to representative case studies from each
category of MWBDs.

###### Index Terms:

Security, privacy, threat modeling, risk assessment, biomedical applications,
next-generation biomedical devices, smart-and-connected health technologies,
miniaturized wireless biomedical devices (MWBDs), injectables, implantables,
ingestibles, wearables, the Internet of Things (IoT), the Internet of Medical
Things (IoMT), the Internet of BioNanoThings (IoBNT), cybersecurity risk,
privacy risk, healthcare system.

##  I Introduction

Rising interest in remote health monitoring and treatment stimulates an
increase in the variety and the volume of miniaturized wireless biomedical
devices (MWBDs) [1, 2, 3, 4]. The need for these smart-and-connected health
technologies is foreseen to remain rising globally as their share in the
reduction of healthcare costs grows [5]. Being convenient, low-cost, and easy-
access, MWBDs support a transition from the traditional reactive medicine to
the proactive, personalized precision healthcare model [6, 7]. We started to
witness more interactive communication between patients and healthcare
providers with the help of remote monitoring devices, including MWBDs,
especially after the global crisis of the public health system caused by the
COVID-19 pandemic [8, 9]. This trend is expected to create even more demand
for MWBDs among patients and healthcare providers [10, 11].

While novel designs are regularly presented, more applications and use
scenarios are envisioned for MWBDs. Therefore, they are still considered
emerging devices. MWBDs may be divided into four main categories:  (1)
injectables, injected underneath the human tissue;  (2) implantables,
implanted into the human body during a surgery;  (3) ingestibles, ingested by
the patient in the form of a regular pill; and  (4) wearables, worn on the
human body.

MWBDs are capable of collecting and transmitting sensitive, private
information, like bio-electrical activity [12, 13] and vital signs [14, 15],
and affecting of the human body through stimulation [16, 17] and drug delivery
[18, 19]. Therefore, while being convenient, they produce privacy and security
risks for their users [20, 21, 22, 23]. Traditionally, designers and
manufacturers of MWBDs tend to prioritize functionality and user experience
over security [21, 22, 23]. As a result, protection mechanisms are missing at
the architectural level for most of MWBDs [13, 14, 15, 16, 17, 18, 19].
Multiple attacks for the misuse of sensitive medical information and the
malfunctioning of MWBDs may be implemented [24]. Professionals are now tasked
with defeating well-funded attacks that, in some cases, can cause immediate
physical harm for the user [25, 26, 27, 28]. Attackers may also be attracted
by the assets belonging to other primary stakeholders of MWBDs, as described
in Table I.

TABLE I: Examples of Potential Risks for Stakeholders of MWBDs Stakeholder | Potential Risk
---|---
User |  (1) The corruption of medical data may lead to wrong diagnoses, and therefore wrong therapies.  (2) Privacy leakage may result in stolen identities and electronic fraud.  (3) Denial of service (DoS) may harm the patient or even cause their death.
Manufacturer |  (1) The leakage of intellectual property (IP) may be used by competitors to increase their market share.  (2) The leakage of IP may also lead to the increase in the number of counterfeits and attacks.  (3) If an attack harms the patient, it may damage the manufacturer’s reputation.  (4) Additionally, lawsuits may be filed against the manufacturer.
Hospital |  If hospitals deal with corrupted MWBDs or counterfeits, they could (1) lose their professional reputation and trust, (2) be involved in lawsuits, or (3) lose their accreditation.
Figure 1: A typical biomedical system. Adapted from [20].

To formulate security objectives efficiently, it is necessary to have a
perspective on the whole system. A typical next-generation biomedical system,
outlined in Fig. 1, consists of an MWBD wirelessly connected to an external
controller. An MWBD usually has a limited functionality:  (1) it may serve as
a controllable actuator, capable of processing a small set of external
instructions from the controller; or  (2) it can operate as a smart sensor,
which transmits the collected medical data to the controller.  The controller
typically acts as a gateway and transfers the user health-related data to a
cloud service for post-processing. The data processed in the cloud is then
sent to a dashboard for the authorized users [20]. The current biomedical
systems implement protection mechanisms mainly beginning from the gateway
[29]. The lack of security in the MWBD puts the whole system in danger.
Therefore, security should be built into MWBDs to protect their users [30].

The MWBDs in the scope of this work typically have at least one network
interface (wireless telemetry) and at least one transducer (sensor or
actuator). These capabilities allow categorizing MWBDs as the Internet of
Things (IoT) devices [7]. The National Institute of Standards and Technology
(NIST) maintains a cybersecurity program dedicated to IoT [31], reflecting the
high demand for security for these devices. Some NIST standards applicable to
MWBDs include NISTIR8200 [7], NISTIR8228 [32], NISTIR8259 [4], and NISTIR8259A
[33]. The developers of MWBDs should also be aware of the drafts from NIST
[34, 35, 36, 37], which will eventually be turned into standards.

While the manufacturers and designers of MWBDs may often not have direct
experience with cybersecurity-related technologies [36], implementating these
mechanisms is of high importance and must be systematic. Accordingly, a
standardized approach to profile potential attackers and to catalog potential
threats is needed for the diverse architectures of emerging MWBDs, as the US
Food and Drug Administration (FDA) announced recently [38]. The process of
understanding, documenting, and evaluating system vulnerabilities, followed by
addressing the protective measures, is known as threat modeling [39]. By using
threat modeling, it is possible to mitigate the weaknesses of systems against
specific adversary scenarios in the early stages of the devices’ life cycles
[40].

In this work, a methodology for modeling and semi-quantitatively assessing
potential threats for the next-generation MWBDs is proposed. This work is
primarily intended to inform the architects and manufacturers of new devices
and pave the way for embedding security in MWBDs. The remainder of this
publication is organized as follows. Section II provides the background on the
security challenges in MWBDs. Section III presents some general considerations
for the threat modeling process. Section IV describes the proposed threat
model for MWBDs. Section V presents an application of this model to real case
studies of the four primary categories of MWBDs. Section VI discusses the
results and makes suggestions for future work. Section VII concludes the work.

##  II Security Challenges for Emerging Miniaturized Wireless Biomedical
Devices

This chapter discusses various security challenges for emerging MWBDs. Some of
these challenges are already present and require immediate attention. Other
challenges are only anticipated in the future. However, both these categories
of challenges illustrate the importance of security for MWBDs.

###  II-A Limited Resources

MWBDs are extremely constrained in area, weight, power, storage, network
interfaces, and computing resources [30]. Therefore, they stand apart from
classical information technology (IT) devices (e.g., smartphones, servers, or
laptops), which have been used to define device cybersecurity capabilities
[4]. The limited resources available to MWBDs bound the range of security
mechanisms applicable to these devices. Such as, the limited area excludes
embedding complex security units occupying a lot of silicon on the chip. The
constrained power excludes complex cryptographic computations and narrows the
communication bandwidth and range. The restricted memory and performance
prevent using sophisticated cryptographic algorithms [24]. Therefore, even
though multiple modern cryptographic algorithms are reliable, the simplicity
of MWBDs makes them unavailable for these devices [4]. Correspondingly,
lightweight cryptographic algorithms, suitable for constrained environments,
need to be developed and standardized [41, 24, 42].

###  II-B Multiple Attack Surfaces

While being restricted in resources, MWBDs are often equipped with diverse
transducers (sensors and actuators), and employ various communication
(network) and power delivery techniques. All these interfaces may be
considered potential channels via which an intruder might maliciously interact
with the device [43]. For a typical MWBD, five such channels may be
identified. Three input channels include the control channel, the sensing
channel, and the power delivering channel. Two output channels include the
user data transferring channel and the actuating channel. Apart from the
attacks on these five external interfaces, additional internal attack surfaces
include the the memory attacks and the digital hardware. An attacker model
covering mentioned channels was presented in [44].

###  II-C Patient’s Safety

Being limited in resources and having multiple attack surfaces, MWBDs should
use protective schemes that would not endanger a patient’s life in an
emergency [24]. Therefore, while these devices require server-side
authentication to ensure that commands are authorized, the critical care
services must be able to access the device even when the normal authentication
method is unavailable. Hence, including the patient into the authentication
schemes, like the one proposed in [45], is potentially dangerous. Also, a
direct disregarding of authentication and authorization in an emergency might
introduce many potential threats. Because of that, authentication in medical
devices remains an open problem [24].

In general, for MWBDs, security and privacy requirements for a device should
not affect its safety, reliability, and resiliency [32]. Traditional IT
security prioritizes confidentiality, integrity, and availability. The ability
of MWBDs to interact with the physical world through sensors and actuators
requires addressing threats to patients and their environments. Depending on
the functionality of a particular biomedical device and its vital necessity
for the patient, availability or integrity may be the highest priority,
followed by privacy and finally confidentiality [7].

###  II-D Distributed Supply Chain

The manufacturing of MWBDs relies on a complex, distributed supply chain. This
chain contains multiple entities, distribution routes, technologies, as well
as diverse legislation and practices. This multifariousness affects the
design, fabrication, distribution, deployment, usage, and maintenance of
MWBDs. Therefore, whether intentionally or unintentionally, the final users of
MWBDs are at risk of supply chain attacks. Supply chain risks for MWBDs may
include the insertion of malicious logic blocks, the use of unauthorized
components and counterfeits, tampering, poor manufacturing and design
practices, etc [46]. Component suppliers often have poor cyber hygiene, and
these vulnerabilities are more of an issue than the ingenuity of the attackers
[25].

###  II-E Lack of Incentives

There is a lack of incentives to build security and privacy into IoT devices.
Cybersecurity is often an afterthought to getting to market, with price and
features prioritized. There is also a general lack of consumer education,
leading to a lack of demand for better cybersecurity and privacy. There are
guidelines available to help manufacturers mitigate risks, but a lack of
incentives to adhere to them [25, 47].

###  II-F Oncoming Challenges

IT innovation is outpacing the development of supporting standards. With the
changing threat environment, the cybersecurity needs of the future should be
considered [7]. One such challenge for cryptography as a whole is that if
large-scale quantum computers are ever built, many current public-key
cryptosystems will be broken [46]. That would compromise the information
security of digital communication. Therefore, NIST initiated a process of
post-quantum cryptography standardization, including quantum-resistant
lightweight algorithms for resource-restricted devices [48, 49].

The list of challenges for MWBDs is not limited by the above examples. Any
influx of new technologies will introduce new security challenges [50], and
new countermeasures should be proposed accordingly. Security is never free and
may cause an extra overhead (e.g., exceeding a tight power budget, increasing
time delays, or causing extra memory usage, etc.) [24]. Considering all the
limitations, it might be adequate to talk about the compromises between
security and other parameters, when these biomedical devices contain, at
least, some basic protective mechanisms against the most possible attacks.
Lightweight security does not mean weak security. However, the lightweight
security properties may be different from those desired for general use: it
may be less robust, less misuse resistant, and have fewer features [51].

The development of protective mechanisms and their embedding into devices
require additional time and financial costs. Yet, ad-hoc security is
insufficient for next-generation biomedical devices [44]. However, if
developers follow well-articulated and transparent principles and practices,
adding secure mechanisms into devices is repeatable [52]. Therefore,
developers should have a guidance for the threat modeling process, allowing
them to estimate and mitigate threats in the early stages of the device life
cycle. In the next section, some general considerations about threat modeling
process will be presented.

##  III Threat Modeling Methodology

Considering the potential effects of cyber-attacks against the emerging MWBDs,
it is necessary to plan for these intrusions and to take steps to prevent them
[53]. Therefore, a high-level method aimed to reveal, document, and address
the security flaws of a system is demanded for these devices. This method is
called threat modeling. Threat modeling uses special security terms, the main
of which are assets, vulnerabilities, threats, attacks, risk, and risk
assessment. Interconnection of these terms is shown in Fig. 2.

Figure 2: Interconnection of terms in threat modeling and risk assessment.
Adapted from [54].

An asset is any item of value present in the system that must be kept secure
and that an adversary aims to steal, modify, or disrupt [39, 44]. A
vulnerability is a weakness in a system caused by a bad design or
implementation. A threat is a circumstance or an event with the potential to
have a malicious effect on assets, individuals, or organizations [55]. An
attack is a malicious activity that attempts to threaten an asset by
exploiting a vulnerability [40]. Risk is a measure of the extent to which an
entity is threatened by a potential circumstance or an event, and typically is
a function of  (1) the adverse impacts that would arise if the circumstance or
event occurs; and  (2) the likelihood of occurrence [55].  Risk assessment is
the process of identifying, estimating, and prioritizing risks to assets,
individuals, or organizations [55]. In particular, qualitative-quantitative
risk assessment is a set of methods, principles, or rules for assessing risk
based on the use of both qualitative terms and associated with them numbers
[55].

Figure 3: Six primary steps of the threat modeling process. Adapted from [39,
55, 40].

Threat modeling is a multistage iterative process that provides insights on
the assets that adversaries may be attracted by and allows recognizing the
most probable attack vectors [56, 57, 40]. The ultimate goal of threat
modeling is to reduce the overall threat risk to an acceptable level. During
threat modeling, all its steps should be collected and organized into a
threat-model document [39, 58, 57]. This document should be kept current,
reflecting new threats and mitigations as they originate [39]. Threat modeling
should be included in the overall development-and-documentation life-cycle
[57]. Setting it apart from the overall design life-cycle may decrease the
number of developers recognizing its importance [39].

The following subsections provide a high-level description of the primary
phases of the threat modeling process, shown in Fig. 3. Activities highlighted
for each phase build on the outcomes of prior activities. The steps provided
for each phase are meant as a starting point and do not entirely define each
activity.

###  III-A Assemble the Threat-Modeling Team

A team that will perform threat modeling should be assembled first. A threat
modeling team should consist of at least one member from each engineering
group (hardware, wireless link, software, and others) to guarantee a complete
understanding of underlying technologies [40].

###  III-B Identify Assumptions and Constraints

The next step in the process is to identify security assumptions and
constraints under which the threat modeling is performed. It allows to capture
the information in an appropriate level of abstraction. These assumptions must
be verified later [39]. This step includes three substeps:

####  III-B1 Analyze the Operational Environment

the team captures the information about the infrastructure and describes the
environment in which risk-based decisions are made. It helps to understand how
different objects and elements in the system (an MWBD, a controller, a user,
medical personnel, and other participants) interact with each other.

####  III-B2 Define Security Domains and Boundaries

in this step, the primary logical components (for example, the analog front-
end, the power management module, the data link, etc.) in the system are
identified. Each logical component may be composed of several physical
components, and have different entry points and threats [39, 43]. Later, these
logical components may be decomposed or merged to achieve a manageable level
of granularity. For example, it may be appropriate to talk about the analog
front-end of a device as a whole, or it may be essential to analyze the
individual functional blocks that form this front-end.

No components are completely trusted, but rather various trust levels may be
assigned to them (for example, high and low trusted components). After the
high and low trusted logical components in the system are identified, the
boundaries and interfaces between them should be determined [40]. After
assumptions about the trust boundaries are made, threat analysis is usually
performed for the data crossing these boundaries. The analysis must consider
the direction of the data moving between trusted and untrusted components.

####  III-B3 Define Use Scenarios

security measures are application-dependent [59]. For each system component,
use scenarios provide a high-level description of how it will be implemented,
deployed, and used. In order to better understand the system behavior, it may
also be useful to list ”anti-scenarios,” which are the settings or the usage
scenarios that are known to be vulnerable or restricted [39, 59].

###  III-C Enumerate Protected Assets

At this stage, assets that will be protected should be identified and listed
for the investigated design. Assets may be tangible (such as, user’s personal
information, therapies, or encryption keys) and intangible (for example, data
consistency, data secrecy, data integrity, or data availability) [60]. The
list of protected assets is later used in the risk analysis. This list for a
particular design must be made considering the perspective of different
stakeholders [44].

###  III-D Define Attackers

Understanding the attacker type is important to understand the resources and
capabilities that they have at their disposal [60]. While real attackers
rarely fit into one category, at a high level, they can be classified based
on:  (1) their position relative to the system (external or internal
adversaries),  (2) their ability to intervene into the system (passive or
active adversaries),  (3) their number (a single entity or a coordinated
group), and  (4) the level of their expertise and equipment (sophisticated or
unsophisticated) [61, 59, 24].

An external intruder is an outside entity that is not a part of the system and
does not have an authorized access [59]. An internal intruder may be:  (a) a
malicious user who performs attacks to learn the secrets of the manufacturer
or to get access to restricted functionality, and  (b) a bad manufacturer who
has the ability to exploit the technology to collect information about the
user or other devices [59] . A passive eavesdropper is capable only to listen
to the communication channel and to get access to the exchanged messages.
These attackers are able to compromise the patient’s privacy. They can
determine if a person has a biomedical device; discover the type of device,
its model and serial number; capture the information about the patient, such
as the ID of their health records, name, age, diagnosis, therapy, and so forth
[24]. An active adversary is not only capable to listen to the channel, but
also to send or replay commands to the device and to modify or block messages.
The motivation for the active attacks may be, for example, to cause
malfunctioning or DoS to the device [24].

###  III-E Reveal Threats

In the next step, threats will be revealed using systematic analysis. For
example, threats can be identified by defining participants (like the user,
the attacker, etc.), their actions, and the consequences of those actions
[39]. The objective is to enumerate the ways by which an attacker can
compromise the system [40].

Threat modeling appears to be more productive when people have an
understanding of how to attack systems [40]. For example, a kill-chain model
[62] studies intrusions from the adversaries’ perspectives by incorporating
the analysis of adversaries, their capabilities, objectives, attitudes, and
limitations. In a kill-chain model, intrusions are described not as singular
events but as phased progressions. This model illustrates that, in fact, the
adversary must successfully move through each stage of the chain to achieve
the desired goal. Therefore, just one mitigation breaks the chain and stops
the adversary [62].

To help people remember the types of threats to which system components might
be exposed [39], the STRIDE model [40] appears to be one of the most known
models [63]. STRIDE is an abbreviation for spoofing, tampering, repudiation,
information disclosure, denial of service, and elevation of privilege [40].
While STRIDE was developed for software systems, and MWBDs are predominantly
hardware systems, some works [56, 57] adopted STRIDE to classify threats for
mobile health systems and IoT devices. Yet, both of these works primarily
focused on the security and privacy of data flows across networks rather than
the specificity of MWBDs.

###  III-F Manage Risks

After the risks for a system are framed, risk management process should
include:

####  III-F1 Assess Risks

risk assessment is a crucial part of effective risk management. It is used to
identify, estimate, and prioritize risks. The purpose of risk assessment is to
inform decision-makers and to plan for risk responses. The result of risk
assessment is a ranked list of threats that reflects the impact of attacks and
the likelihood that harm will occur [55].

Risk and its contributing factors can be assessed in a variety of ways,
including quantitatively, qualitatively, or semi-quantitatively [55]. While
both quantitative and qualitative assessments have their limitations, the
semi-quantitative assessment provides the benefits of both these approaches.
This method typically employs bins, scales, or representative numbers. Bins or
scales translate easily into qualitative terms and also allow relative
comparisons between values. The role of expert judgment in assigning values is
more evident than in a purely quantitative approach. Also, when scales or sets
of bins provide enough granularity, relative prioritization among results is
better supported than in a purely qualitative approach [55].

The DREAD model [40] was proposed to assess the associated risks for specific
threats in mobile health systems [56]. DREAD is an acronym for damage
potential, reproducibility, exploitability, affected users, and
discoverability. However, this model was initially designed to rank errors,
flaws, or faults in software [40]. Since MWBDs are predominantly hardware
systems, this model is not entirely suitable for them. For example, in the
case of MWBDs, there is only one user, and therefore the criteria of affected
users would not contribute to the final decision. The category of
discoverability also seems not to be informative as it is difficult to
estimate and is usually set equal to the maximum possible value [40].

The smartcard community introduced the guidance metrics to calculate the total
effort required by an attacker to perform a successful attack. This guidance
is described in the CCDB-2009-03-001, Common Criteria ”Application of Attack
Potential to Smartcards” [64]. This document aims to be applied to the
operational behavior of a smartcard and applications specific only to hardware
or software. Therefore, since MWBDs are mostly hardware systems, while some of
them also have processing units running software, this document could serve as
a base for the methodology to efficiently evaluate the threats for MWBDs.

####  III-F2 Respond to Risks

in this step, the corresponding techniques and technologies should be chosen
to respond to the discovered threats. Depending on the threat model,
customers, and expected use cases, various countermeasures may be proposed
[65, 33]. As a starting point for MWBDs, the model for the lightweight
implementation of data security may be applied [44]. While providing a
guideline on how to start protecting MWBDs, this model primarily focuses on
data security. Therefore, other interfaces (sensing channel, power delivering
channel, and actuating channel) require a separate analysis of available
protective mechanisms. In general, engineers should weigh the value of each
security countermeasure for MWBDs to reach a trade-off between safety,
reliability, resilience, security, and privacy risks.

####  III-F3 Monitor Risks

risk assessment is not simply a one-time activity that provides permanent and
definitive information for decision-makers. Monitoring risk factors (threats,
vulnerabilities, capabilities and intent of adversaries, etc.) over time can
provide critical information on changing conditions that could potentially
affect the security of systems. Information derived from the ongoing risk
monitoring can be used to refresh risk assessments [55].

In the next chapter, a specific threat model for emerging MWBDs will be
introduced. According to the described threat modeling methodology,
assumptions about the operational environment, security boundaries, and use
scenarios will be identified first. Then, suggestions about protected assets
and attackers will be provided. Finally, a risk assessment methodology will be
proposed.

##  IV Proposed Threat Model for MWBDs

Figure 4: General architecture of an MWBD. Attack points are shown in red.
Adapted from [47].

While at least a dozen of threat modeling methods exist in IT [63], they have
mostly been developed for either purely software systems or networking
systems. However, MWBDs are predominantly hardware systems, and these existing
threat modeling techniques may be less efficient for them. Therefore,
describing a specific threat model for emerging MWBDs would help developers to
make their systems more secure.

The generic models for implementing security in MWBDs, like the one proposed
in [44], may be a good starting point. However, each particular device would
require performing a separate threat analysis. The design of a threat model
requires security expertise. After the threat model is defined, the threat
analysis becomes an engineering task and may be performed by non-experts in
security [40].

We propose a domain-specific threat model that aims to help the designers and
manufacturers of MWBDs to identify threats and embed security in their designs
in the pre-market phase of the lifecycle of an MWBD. While some of the
proposed threat modeling process components may or may not be applicable for a
particular MWBD, the overall process is valid for a wide range of devices.

###  IV-A Assumptions

####  IV-A1 Operational Environment

this model considers a single victim using an MWBD in a public space,
accessible to multiple people, including but not limited to adversaries. Being
in a public space, attackers can neither have physical access to the user
(device) nor utilize large high-end equipment. For each particular case study
of MWBDs, additional assumptions about its operational environment may be
required.

####  IV-A2 Security Domains and Boundaries

in general, an MWBD controls and monitors some physical process (a health
condition) in the human body. A set of sensors report the state of this health
condition to the processor. Based on the information received from sensors,
the processor defines the control signals to actuators to maintain the desired
state. The processor often communicates with an external controller that
monitors or configures the device. This communication is performed via a
wireless telemetry interface (data/control link). The power is typically also
delivered wirelessly, except in more complex battery-powered wearables or
implantables, like pacemakers. The processor may store various sensitive data
(end users’ private data, the chip ID, etc.) in the off-chip memory. This
general architecture, presented in Fig. 4, considers eleven attack points, for
which examples are provided in Table II.

TABLE II: Attack Points - Examples of Attacks Attack point | Attack example
---|---
1 |  Fault injection attack [66].
2, 4, 6, 8 |  Probing [67].
3 |  Hardware trojans [68].
5 |  Control spoofing [69].
7 |  Microprobing [70].
9 |  Shielding / cutting the antenna.
10 |  |  Denial of sleep [71]. Power Analysis [72].
---
11 |  Man-in-the-middle (MIM) [73].

This work focuses on non-invasive direct attacks because the direct channels,
missing security mechanisms, should be protected in the first place. Among the
eleven defined attack points, direct interfaces include sensors (attack point
1), actuators (attack point 5), and telemetry (attack points 9, 10, and 11).
For MWBDs, sensors and actuators are mainly located on/in the body. Therefore,
it is hard to access them once they are deployed. Also, before deployment,
there is a rigorous calibration for MWBDs, which makes it harder to deploy a
tampered device. Therefore, for direct channels, attacks against telemetry
(wireless data and power transfer) are prioritized in this work. Attack points
2 - 4 and 6 - 8 are out of scope for this model. Securing the remaining
elements of the biomedical system (the physical process, the controller, the
cloud, and the dashboard) needs separate analysis and is out of scope for this
study.

To have a manageable level of complexity in the model, we consider that
integrated circuits (ICs) are trustable after fabrication and testing, which
means that malicious logic blocks are not inserted into the design (a chip
manufacturer is trustable). After deployment, the IC design is fixed, in other
words, there are no dynamic attacks on hardware. Semi-invasive (chip imaging,
laser probing, voltage contrast, photo-emission, etc.) and invasive (reverse
engineering, laser fault injection, etc.) hardware attacks are out of scope
for this work.

TABLE III: Risk Assessment - Characteristics and Scales

| Characteristic | QLV |  | QNV
---
Description
|  |  Expert | 1 |  | Broad expertise in cybersecurity. Familiar with the target device at the developer level. Experienced
---
with, and equipped by, sophisticated tools, for which the expertise in using
is difficult to obtain.
|  |  Proficient | 2 |  |  Familiar with security behavior, classical attacks [64], and related disciplines (electrical engineering,
---
software development, etc.).
|  C1: Expertise of the attacker |  Layman | 3 | No particular expertise.
|  |  Custom | 1 | Bespoke equipment.
|  |  Specialized | 2 |  | Expensive commercially available equipment. Sales are controlled by manufacturers. The expertise
---
in using the equipment is difficult to obtain. For example, the type of
expensive equipment which
universities have in their possession [64].
|  C2: Equipment required to carry out the attack |  Standard | 3 |  | Mass-market commercially available equipment. The expertise in using the equipment may be
---
acquired from publicly available resources. For example, smartphones or
laptops.
|  |  Nearby | 1 |  | The attacker is in close proximity (the same room) to, and in immediate visibility to, the victim.
---
No physical obstacles, like walls and doors, exist between the attacker and
the victim (device).
|  |  Moderate | 2 |  | The attacker is in the same space with the victim, there are no physical obstacles between the attacker
---
and the victim, but the distance between the attacker and the victim does not
allow the victim to see
the attacker.
|  C3: Physical proximity to the attacked device |  Remote | 3 | The attacker is capable to mount an attack while in a different location than the victim.
|  |  Long | 1 | The attacker is able to access the device continuously.
|  |  Moderate | 2 | The attacker is able to access the device multiple times.
|  C4: Device access time |  Short | 3 | The attacker is able to access the device once in real time.
|  |  Critical | 1 |  Low-level information about hardware design or source code [64].
|  |  Restricted | 2 |  Proprietary confidential developer’s information like specifications or guidances [64].
Probability |  C5: Device information |  Public | 3 | Public domain information.
|  |  High | 3 |  | Attacks have a severe or catastrophic adverse effect on the user. In some circumstances, DoS attacks
---
may lead to an irreparable harm, such as apoplexy, or even a loss of human
life.
|  |  Moderate | 2 |  | Attacks have a moderate adverse effect on the user. For example, alternating the sensor data may cause
---
wrong commands to actuators, directly impacting the health of the victim, but
the malicious effect is
temporary.
Impact |  C6: Severity of the attack |  Low | 1 |  | Attacks have a limited adverse effect on the user (pose surmountable problems for the victim). For
---
example, under certain context, loss of personal information do not prevent
the MWBD from its correct
functioning.
Note: QLV and QNV stand for a qualitative value and a quantitative value
accordingly.

####  IV-A3 Use Scenarios

use scenarios are unique for each device, and therefore cannot be generalized.
Each particular MWBD would require listing its use scenarios. To give a clear
picture to the readers through examples, Section V of this work provides case
studies for each category of MWBDs, namely injectables, implantables,
ingestibles, and wearables.

###  IV-B Protected Assets

Among various stakeholders of MWBDs, this model focuses on the user. This
perspective requires balancing safety, service availability, resilience, and
privacy. Safety protects from hazards, risks, or injury caused by the
operation of the device. Service availability protects against denial of
device service. Resilience means security against most attacks and the ability
to return to a safe state in case of a successful attack. Privacy means
protecting the confidentiality and integrity of personally identifiable
information (PII). Privacy goals include:  (1) device-existence privacy;  (2)
device-type privacy;  (3) unique device ID privacy;  (4) measurement and log
privacy;  (5) patient privacy; and  (6) patient location privacy [61].
Confidentiality prevents the improper disclosure of information, and data
integrity prevents the improper modification of information.

Assets are unique for each MWBD. Therefore, each case study of MWBDs would
require specific analysis. After the assets are listed, attacks for each of
them should be defined.

###  IV-C Attackers

This model considers sophisticated attackers, who have the intent and
capabilities to attack the MWBD. The attacker may be both an individual
(outsider, insider, trusted insider, or privileged insider) and an established
group. The attacker may be both passive and active. The adversary, however,
does not have physical access to the user of the MWBD, and hence all attacks
are remote. For this reason, attacks on the physical process are not
considered in the proposed model. However, for the smartwatch-like wearables,
the attacker may manipulate the device for a limited period of time before the
user begins to use it (if the device was left unattended after deployment).
Large high-end equipment is excluded from consideration in this paper since an
attacker is not able to bring it to a public place.

TABLE IV: Case Studies - Devices and Assets

Number | Name | Category | Purpose | Status | Assets | Ref.
---|---|---|---|---|---|---
D1 | BioMote | Injectable |  | A wireless sensor node for
---
continuous monitoring of the
blood alcohol content
(ethanol, background, and pH).
| In-vitro
---
tests.
| 1\. Backscattered user sensor data
---
(blood alcohol content).
[14]
D2 |  | Wireless
---
Capsule Endoscope
Ingestible |  | A wireless spherical endoscopic
---
capsule for ColoRectal Cancer (CRC)
screening with a locomotion control.
| In-vitro
---
tests.
| 1\. User data (cancer information,
---
video frames)
2\. Control signals from an external
device.
[74]
D3 |  | Trimodal wireless
---
implantable
neural interface
System-on-Chip (SoC)
Implantable |  | A wireless trimodal neural interface
---
SoC, providing optical/electrical
stimulation capabilities and neural
recordings.
| In-vivo
---
tests
in freely
behaving
animals.
| 1\. Recording/stimulation
---
parameters (BLE and on-off-keying
(OOK) signals at 13.56 MHz).
2\. Evoked neural activities
(OOK RF signals at 433 MHz).
[75]
D4 |  | An integrated readout
---
circuit for a transcutaneous
oxygen sensing wearable
device
Wearable |  | A fluorescence-based readout
---
dedicated to sensing transcutaneous
oxygen diffusing through the skin.
| Ex-vivo
---
tests.
| 1\. User data (partial pressure
---
of transcutaneous $O_{2}$).
2\. Control signals from
an external controller.
[76]

###  IV-D Risk Assessment

In the proposed model, six relevant characteristics of threats are selected to
assess risks for an MWBD (see Table III):

  1. 1.

Expertise of the attacker reflects the extent to which related knowledge is
necessary for the adversary to perform a successful attack.

  2. 2.

Equipment required to carry out the attack describes the tools that an
adversary needs to use to carry out an attack.

  3. 3.

Physical proximity to the attacked device shows how close the adversary should
be to the user of an MWBD to mount a successful attack.

  4. 4.

Device access time evaluates the time during which the attacker can have
access to the attacked device.

  5. 5.

Device information evaluates the need for the particular information assisting
the attack, which cannot be substituted by a related combination of time and
expertise [64].

  6. 6.

Severity of the attack estimates the loss caused by its occurrence. In this
model, severity corresponds to the physical harm for the user caused by a
successful attack.

For each characteristic $C_{1}-C_{6}$ in Table III, a three-tiered
qualitative-quantitative scale is assigned. While this scale may provide
limited granularity, it makes the first iteration of the proposed model less
complex and easier to apply. Additional tiers may be added when required.

Characteristics $C_{1}-C_{5}$ in Table III define the probability of an
attack. The quantitative values for these categories are assigned in the
reverse order: the fewer efforts are needed to perform an attack, the higher
its likelihood, and therefore the higher the corresponding score. The total
probability $P$ of an attack equals the sum of the values for $C_{1}-C_{5}$.
The impact of an attack $I$ equals the value of the characteristic $C_{6}$ in
Table III, for which the highest score corresponds to the highest severity.

After potential threats are identified, and their principal characteristics
are captured, the total risk $R$ for each of them may be defined based on
their impact $I$ and probability $P$. In risk management, the risk matrix
approach is a typical qualitative-quantitative tool to evaluate various risks.
Even though it is not mathematically rigorous, its visibility and ease of
application make it well-received in various industries [77].

TABLE V: Risk matrix | Impact (I)
---|---
Probability (P) | Low (1) | Moderate (2) | High (3)
Low (5-7) | Very Low | Moderate | High
Moderate (8-12) | Low | Moderate | High
High (13-15) | Moderate | High | Very High

For this model, the risk matrix shown in Table V is used to assess risks,
where different colors code different levels of risk. After the risk
assessment is completed for each threat and the data are filled in the risk
matrix, the results appear sorted according to their risk levels. While these
results suggest which threats are perilous and require more attention,
designers should decide which threat to address first, based on their
abilities and specific requirements for the design.

##  V Case Studies for Emerging Miniaturized Wireless Biomedical Devices

TABLE VI: Case Studies - Threats

|  |  |  | Probability | Impact
---|---|---|---|---|---
Number | Threat | Threat violates | Device | C1 | C2 | C3 | C4 | C5 | Total | C6
T1 |  | The attacker uses a counterfeit wearable controller to power up the
---
device and to collect the sensitive private health data. As a result,
the confidentiality of the patient’s personal information is violated.
| Confidentiality
---
Authenticity
D1 | 3 | 3 | 2 | 3 | 3 |  | 14
---
(High)
| 1
---
(Low)
T2 |  | The attacker conducts a MIM attack using special equipment to tamper
---
with the data / control signals, producing a false report about the health
condition / a false command, causing a false treatment or therapy. This
may result in temporary or permanent health damage. Even if the device
does not have actuators, a physician working with the corrupted sensor
data can prescribe a wrong treatment or therapy.
Integrity |  | D1
---
D3
1 | 2 | 2 | 3 | 3 |  | 11
---
(Moderate)
| 3
---
(High)
T3 |  | The attacker jams the wireless data link. Sensor data cannot be
---
collected accurately, and stimulation cannot be applied correctly.
This may result in permanent health damage due to the incorrect
or missing treatment.
Availability |  | D1
---
D3
1 | 2 | 3 | 2 | 3 |  | 11
---
(Moderate)
| 3
---
(High)
T4 |  | The attacker uses a software-defined radio or an external hub to
---
collect the data about the evoked neural activities of the user. This
leads to a leak of the patient’s personal confidential information.
| Confidentiality
---
Authenticity
D3 | 2 | 3 | 2 | 3 | 3 |  | 13
---
(High)
| 1
---
(Low)
T5 |  | The attacker eavesdrops the data, using the standard Bluetooth
---
equipment. This leads to a leak of the patient’s confidential
information.
| Confidentiality
---
Authenticity
| D2
---
D4
2 | 2 | 2 | 3 | 3 |  | 12
---
(Moderate)
| 1
---
(Low)
T6 |  | The attacker interferes with the communication channel and
---
substitutes the user data by some counterfeit data. This may result
in permanent health damage due to a wrong or missing therapy.
Authenticity |  | D2
---
D4
1 | 2 | 2 | 3 | 3 |  | 11
---
(Moderate)
| 3
---
(High)
T7 |  | The attacker replays a command to decrease the illumination or
---
to switch the Bluetooth module off to decrease the amount of
information that can be extracted from video frames. This would
require to repeat the measurements with a different device.
Availability | D2 | 2 | 2 | 2 | 3 | 3 |  | 12
---
(Moderate)
| 1
---
(Low)
|  |  | D2 |  |  |  |  |  |  |  | 1
---
(low)
T8 |  The attacker sends a high-volume radio traffic to deplete the battery. Even if the device supports authentication, the process of commands’ and data validation would consume extra power, which could lead eventually to denial of service. | Availability | D4 | 2 | 2 | 2 | 3 | 3 |  12 (Moderate) |  | 3
---
(High)
| Note: for T8, D2 and D4 have different impacts. D2 is used for short-term
monitoring of the digestive tract. Measurements are performed in a laboratory
under the
---
supervision of a physician. In this case, the denial of service may be quickly
resolved and the procedure may be repeated with another device. However, D4
can be used
outside the hospital. In the event of DoS, it may not be quickly replaced with
another device. Therefore, the impact of T8 is higher for D4.

This chapter aims to provide meaningful examples of MWBDs (see Table IV) and
their associated threats (see Table VI), which are disclosed and prioritized
with the application of the proposed threat model. For each category of MWBDs,
one representative device was selected. At the time of writing this paper,
most MWBDs were either at the proof-of-concept stage or the stage of pre-
clinical trials in freely-behaving animals. Nevertheless, since they have a
potential for large-scale manufacturing and are intended to be ultimately used
in humans, it is of interest to analyze threats for them using the proposed
threat model.

The following subsections will provide brief overviews of these case studies,
including information about their purposes, internal structure, and
operational environments. For each of them, risk assessment results will be
presented in the form of risk matrices. Since the primary purpose of this
chapter is to illustrate the application of the designed model but not to
perform a comprehensive threat modeling for the selected devices, it does not
guarantee to include all possible threats for the selected case studies.

###  V-A Case study 1 - Injectable

Figure 5: Device 1: block diagram. Adapted from [14].

BioMote [14], shown in Fig. 5 and further referred to as device 1 (D1), is a
wireless sensor node intended for continuous monitoring of the blood alcohol
content. D1 is routinely used outside of a clinical laboratory. D1 is
subcutaneously injected into the interstitial fluid and wirelessly paired with
an external wearable controller. The D1’s electrochemical sensor array
measures alcohol and pH. These measurements are unidirectionally transmitted
to the controller through backscatter using a current-to-frequency converter.
D1 is wirelessly powered by the controller via an inductive link.

TABLE VII: Risk matrix - Device 1 | Impact (I)
---|---
Probability (P) | Low | Moderate | High
Low | – | – | –
Moderate | – | – |  | T2, T3
---
High | T1 |  | –
---
–

For D1, the risk matrix is presented in Table VII. It shows that T1 has a
moderate risk while T2 and T3 have high risks.

###  V-B Case study 2 - Ingestible

Figure 6: Device 2: (a) 3D design. (b) Block diagram. Adapted from [74].

A wireless capsule endoscope [74], shown in Fig. 6 and further referred to as
device 2 (D2), is intended for ColoRectal Cancer (CRC) screening with
locomotion control. The primary modules of D2 include an image sensor with
optics, an illumination board, a control unit, a telemetry board, an actuation
system, a localization unit, and a battery with a recharging circuit. D2 is
used in a clinical laboratory and swallowed by the patient. The image sensor
of D2 captures the condition of the patient’s digestive tract. The collected
images are streamed via Bluetooth to the external controller.

TABLE VIII: Risk matrix - Device 2 | Impact (I)
---|---
Probability (P) | Low | Moderate | High
Low | – | – | –
Moderate | T5, T7, T8 | – |  | T6
---
High | – |  | –
---
–

For D2, the risk matrix is presented in Table VIII. It demonstrates that,
among four detected threats, T6 has a high risk and T5, T7, and T8 have low
risks.

Figure 7: Device 3: block diagram. Adapted from [75].

###  V-C Case study 3 - Implantable

A trimodal wireless implantable neural interface SoC [75], shown in Fig. 7 and
further referred to as device 3 (D3), provides optical/electrical stimulation
capabilities and neural recordings. D3 consists of optical and electrical
stimulation blocks, an analog front-end, a data packetizer, telemetry blocks,
a timing control unit, and a power management module. D3 is used in a clinical
laboratory. D3 is implanted into the brain and wirelessly paired with a
control arena. The recording/stimulation parameters are sent to the arena via
BLE from an external terminal (computer). The arena relays the BLE parameters
to D3 by on-off-keying (OOK) of a 13.56 MHz power carrier via inductive coils.
D3 generates optical/electrical stimulation pulses based on the received
parameters. The evoked neural activities are sensed, processed, and
transmitted by D3 to the terminal by OOK at 433 MHz. The terminal receives the
data by a pair of software-defined radios (SDRs). D3 is not currently used in
humans but for scientific experiments in freely behaving animals (rodents).
However, it was selected as a case study since its functionality is similar to
that of commercial products like [78]. In addition, the commercial products
themselves are proprietary, which makes them unavailable for the analysis
based on public data.

TABLE IX: Risk matrix - Device 3 | Impact (I)
---|---
Probability (P) | Low | Moderate | High
Low | – | – | –
Moderate | – | – |  | T2, T3
---
High | T4 |  | –
---
–

For D3, the risk matrix is presented in Table IX. It shows that T2 and T3 have
high risks, whereas T4 has a moderate risk.

###  V-D Case study 4 - Wearable

Figure 8: Device 4: block diagram of the readout IC. Adapted from [9].

An integrated readout circuit [76], shown in Fig. 8 and further referred to as
device 4 (D4), is used for non-invasive transcutaneous oxygen sensing, which
correlates with the blood oxygen level. D4 primary blocks include an analog
front-end, a light-emitting diode (LED) driver, and a power management block.
D4 is routinely used both in a home setting and in a clinical environment. D4
is intended to be used with dry electrodes in the form of a smart watch or a
smart patch. D4 is projected to be a battery-powered device transmitting the
sensor data to the external controller via BLE.

TABLE X: Risk matrix - Device 4 | Impact (I)
---|---
Probability (P) | Low | Moderate | High
Low | – | – | –
Moderate | T5 | – |  | T6, T8
---
High | – |  | –
---
–

For D4, the risk matrix is presented in Table X. It reveals that T6 and T8
have high risks while T5 has a low risk.

##  VI Discussion and Future Work

The use of the proposed model revealed and prioritized threats for the case
studies of injectables, ingestibles, implantables, and wearables, showing that
the model is applicable for a wide range of devices. However, the proposed
threat model enables performing further validation, which would involve
additional case studies. This validation may be done by investigating known
vulnerabilities in devices and comparing the results of the analysis with the
outcome of other models. It would also be of interest to include more
commercial devices. However, being proprietary and closed source, these
devices are challenging to be analyzed based on the public domain information
[61]. Based on the results of these additional investigations, it may become
apparent if separate threat models for each category of MWBDs may provide more
information for designers and manufacturers. Another suggestion for future
work is to design threat models focused on other primary stakeholders of
MWBDs, including manufacturers and hospitals.

##  VII Conclusion

This work discussed the importance of security for the emerging miniaturized
wireless biomedical devices. The combination of valuable assets belonging to
different stakeholders and multiple attack surfaces makes MWBDs a target for
cybercriminals. Since MWBDs pose significant risks for their stakeholders,
security should be embedded into MWBDs in a structured and repeatable way
during the pre-market phase. The initial step in embedding security into a
design is to perform threat modeling. However, it has been shown that MWBDs
are distinct from conventional IT devices and require a unique threat model.
Therefore, first, this work described the threat modeling process for MWBDs.
Then, a domain-specific qualitative-quantitative threat model, suitable for a
wide range of MWBDs, was proposed. Among various stakeholders, this model
focused on the user.

The model suggested to use six relevant characteristics of attacks to assess
their probability and impact. For each characteristic, a three-tiered
qualitative-quantitative scale was assigned. The total risk of an attack was
defined using the risk matrix approach. The threat model was then applied to
the representative designs from each main category of MWBDs. The primary
intent of case studies was to detect several threats to show how the model may
be adopted by a threat modeling team. The outcomes of the risk analysis reveal
that the analyzed devices are vulnerable. The model was easy to apply and
sufficient to describe all the observed threats.

## References

  * [1] C. J. Bettinger, “Advances in materials and structures for ingestible electromechanical medical devices,” _Angewandte Chemie International Edition_ , vol. 57(52), pp. 16 946–16 958, Dec. 2018.
  * [2] H. C. Koydemir and A. Ozcan, “Wearable and implantable sensors for biomedical applications,” _Annual Review of Analytical Chemistry_ , vol. 11, pp. 127–146, Jun. 2018.
  * [3] J. Dunn, R. Runge, and M. Snyder, “Wearables and the medical revolution,” _Personalized medicine_ , vol. 15(5), pp. 429–448, Sep. 2018.
  * [4] _Foundational cybersecurity activities for IoT device manufacturers_ , NISTIR 8259, 2020.
  * [5] A. Meola, “IoT healthcare in 2021: companies, medical devices, and use cases,” Business Insider, Feb. 2021. [Online]. Available: https://www.businessinsider.com/iot-healthcare. [Accessed Apr.22, 2021].
  * [6] A. Kiourti and K. S. Nikita, “A review of in-body biotelemetry devices: Implantables, ingestibles, and injectables,” _IEEE Transactions on Biomedical Engineering_ , vol. 64(7), pp. 1422–1430, Feb. 2017.
  * [7] _Interagency report on the status of international cybersecurity standardization for IoT_ , NISTIR 8200, 2018.
  * [8] T. Weil and S. Murugesan, “IT risk and resilience – cybersecurity response to COVID-19,” _IT Professional_ , vol. 22(3), May 2020.
  * [9] U. Guler, I. Costanzo, and D. Sen, “Emerging blood gas monitors: how they can help with COVID-19,” _IEEE Solid-State Circuits Magazine_ , vol. 12(4), pp. 33–47, Nov. 2020.
  * [10] S. Allen, “2020 US and global health care outlook,” Deloitte, 2020\. [Online]. Available: https://www2.deloitte.com/us/en/pages/life-sciences-and-health-care/articles/global-health-care-sector-outlook.html#. [Accessed Apr.22, 2021].
  * [11] J. E. Hollander and B. G. Carr, “Virtually perfect? Telemedicine for COVID-19,” _New England Journal of Medicine_ , vol. 382(18), pp. 1679–1681, Apr. 2020.
  * [12] S. Song et al., “A 769$\mu$W battery-powered single-chip SoC with BLE for multi-modal vital sign monitoring health patches,” _IEEE Transactions on Biomedical Circuits and Systems_ , vol. 13(6), pp. 1506–1517, Oct. 2019.
  * [13] M. Li, W. Xiong, and Y. Li, “Wearable measurement of ECG signals based on smart clothing,” _International Journal of Telemedicine and Applications_ , vol. 2020, Jan. 2020.
  * [14] H. Jiang et al., “A sub-1$\mu$W multiparameter injectable BioMote for continuous alcohol monitoring,” in _2018 IEEE Custom Integrated Circuits Conference (CICC)_ , Apr. 2018, pp. 1–4.
  * [15] M. Mimee et al., “An ingestible bacterial-electronic system to monitor gastrointestinal health,” _Science_ , vol. 360, no. 6391, pp. 915–918, May 2018.
  * [16] J. Charthad et al., “A mm-sized wireless implantable device for electrical stimulation of peripheral nerves,” _IEEE Transactions on Biomedical Circuits and Systems_ , vol. 12(2), pp. 257–270, Mar. 2018.
  * [17] B. C. Johnson et al., “StimDust: a 6.5$mm^{3}$, wireless ultrasonic peripheral nerve stimulator with 82% peak chip efficiency,” in _2018 IEEE Custom Integrated Circuits Conference (CICC)_ , 2018, pp. 1–4.
  * [18] B. Yan et al., “Battery-free implantable insulin micropump operating at transcutaneously radio frequency-transmittable power,” _Medical Devices & Sensors_, vol. 2(5-6), p. e10055, Oct. 2019.
  * [19] X. Guo et al., “A novel and reproducible release mechanism for a drug-delivery system in the gastrointestinal tract,” _Biomedical microdevices_ , vol. 21(1), pp. 1–9, Mar. 2019.
  * [20] G. Selimis, “A healthy approach to medical security,” _Electronics Europe News_ , pp. 30–31, Jan. 2020.
  * [21] W. Sun et al., “Security and privacy in the medical internet of things: a review,” _Security and Communication Networks_ , vol. 2018, Mar. 2018.
  * [22] B. Alexander, S. Haseeb, and A. Baranchuk, “Are implanted electronic devices hackable?” _Trends in cardiovascular medicine_ , vol. 29, no. 8, pp. 476–480, Nov. 2019.
  * [23] D. Kotz et al., “Privacy and security in mobile health: a research agenda,” _Computer_ , vol. 49(6), pp. 22–30, Jun. 2016.
  * [24] C. Camara, P. Peris-Lopez, and J. E. Tapiador, “Security and privacy issues in implantable medical devices: a comprehensive survey,” _Journal of biomedical informatics_ , vol. 55, pp. 272–289, Jun. 2015.
  * [25] _IoT cybersecurity colloquium: a NIST workshop proceedings_ , NISTIR 8201, 2017.
  * [26] L. H. Newman, “A new pacemaker hack puts malware directly on the devices,” Wired, Aug. 2018. [Online]. Available: https://www.wired.com/story/pacemaker-hack-malware-black-hat/. [Accessed Apr.22, 2021].
  * [27] S. Shin and J. Lipton, “Security researchers say they can hack Medtronic pacemakers,” CNBC, Aug. 2018. [Online]. Available: https://www.cnbc.com/2018/08/17/security-researchers-say-they-can-hack-medtronic-pacemakers.html. [Accessed Apr.22, 2021].
  * [28] F. Donovan, “Medtronic Criticized for Lax Medical Device Security Response,” Health IT Security, Aug. 2018. [Online]. Available: https://healthitsecurity.com/news/medtronic-criticized-for-lax-medical-device-security-response. [Accessed Apr.22, 2021].
  * [29] S. Tuli et al., “Next generation technologies for smart healthcare: challenges, vision, model, trends and future directions,” _Internet Technology Letters_ , p. e145, Mar. 2019.
  * [30] P. A. Williams and V. McCauley, “Always connected: the security challenges of the healthcare Internet of Things,” in _2016 IEEE 3rd World Forum on Internet of Things (WF-IoT)_ , Dec. 2016, pp. 30–35.
  * [31] NIST cybersecurity for IoT program, 2016. [Online]. Available: https://www.nist.gov/programs-projects/nist-cybersecurity-iot-program. [Accessed Jan.17, 2021].
  * [32] _Considerations for managing Internet of Things (IoT) cybersecurity and privacy risks_ , NISTIR 8228, 2019.
  * [33] _IoT device cybersecurity capability core baseline_ , NISTIR 8259A, 2020.
  * [34] _IoT device cybersecurity guidance for the federal government: establishing IoT device cybersecurity requirements_ , NIST SP 800-213 (Draft), 2020.
  * [35] _IoT non-technical supporting capability core baseline_ , NISTIR 8259B (Draft), 2020.
  * [36] _Creating a profile using the IoT core baseline and non-technical baseline_ , NISTIR 8259C (Draft), 2020.
  * [37] _Profile using the IoT core baseline and non-technical baseline for the federal government_ , NISTIR 8259D (Draft), 2020.
  * [38] M. K. McGee, “FDA’s Kevin Fu on threat modeling for medical devices,” Careers Info Security, 2020. [Online]. Available: https://www.careersinfosecurity.com/interviews/fdas-kevin-fu-on-threat-modeling-for-medical-devices-i-4869. [Accessed Apr.22, 2021].
  * [39] P. Torr, “Demystifying the threat modeling process,” _IEEE Security & Privacy_, vol. 3(5), pp. 66–70, Oct. 2005.
  * [40] M. Howard and D. Leblanc, _Writing Secure Code_ , 2nd ed., Redmond, WA, Microsoft Press, 2002.
  * [41] M. Alioto and S. Taneja, “Enabling ubiquitous hardware security via energy-efficient primitives and systems,” in _2019 IEEE Custom Integrated Circuits Conference (CICC)_ , Apr. 2019, pp. 1–8.
  * [42] NIST Lightweight Cryptography Project, 2016. [Online]. Available: https://csrc.nist.gov/Projects/lightweight-cryptography. [Accessed Jan.17, 2021].
  * [43] J. Di and S. Smith, “A hardware threat modeling concept for trustable integrated circuits,” in _2007 IEEE Region 5 Technical Conference_. IEEE, Apr. 2007, pp. 354–357.
  * [44] V. Vakhter et al., “Minimum on-the-node data security for the next-generation miniaturized wireless biomedical devices,” in _2020 IEEE 63rd International Midwest Symposium on Circuits and Systems (MWSCAS)_ , Aug. 2020, pp. 1068–1071.
  * [45] S. Maji et al., “A low-power dual-factor authentication unit for secure implantable devices,” in _2020 IEEE Custom Integrated Circuits Conference (CICC)_ , Mar. 2020, pp. 1–4.
  * [46] _2019 NIST/ITL cybersecurity program annual report_ , SP 800-211, 2020.
  * [47] A. Cardenas, “Cyber-physical systems security knowledge area,” London, UK, CyBOK, 2019.
  * [48] NIST post-quantum cryptography project, 2017. [Online]. Available: https://content.csrc.e1a.nist.gov/Projects/Post-Quantum-Cryptography. [Accessed Apr.2, 2021].
  * [49] NIST reveals 26 algorithms advancing to the post-quantum crypto ‘semifinals’, May 2019. [Online]. Available: https://www.nist.gov/news-events/news/2019/01/nist-reveals-26-algorithms-advancing-post-quantum-crypto-semifinals. [Accessed Apr.22, 2021].
  * [50] J. M. Franklin et al., “Security analysis of first responder mobile and wearable devices,” NIST, May 2020. [Online]. Available: https://www.nist.gov/publications/security-analysis-first-responder-mobile-and-wearable-devices. [Accessed Feb.2, 2021].
  * [51] K. McKay, “Introduction to cryptographic work at NIST,” Worcester Polytechnic Institute - WPI, Cybercorps. Jan. 30, 2020.
  * [52] G. Hunt, G. Letey, and E. Nightingale, “The seven properties of highly secure devices,” Tech. Report MSR-TR-2017-16, Mar. 2017.
  * [53] C. Martin, “Cybersecurity and COVID-19,” _Computing Edge_ , vol. 6(10), 2020\.
  * [54] B. Sanz et al., “A threat model approach to attacks and countermeasures in on-line social networks,” in _11th Reunion Espanola de Criptografıa y Seguridad de la Información (RECSI)_ , Aug. 2011, pp. 343–348.
  * [55] _Guide for conducting risk assessments_ , NIST SP 800-30 Rev.1, 2012.
  * [56] Cagnazzo, Matteo et al., “Threat modeling for mobile health systems,” in _2018 IEEE Wireless Communications & Networking Conference Workshops_, Apr. 2018, pp. 314–319.
  * [57] Seeam, Amar et al., “Threat modeling and security issues for the Internet of Things,” in _2019 Conference on Next Generation Computing Applications (NextComp)_ , Sep. 2019, pp. 1–8.
  * [58] “Five steps to successful threat modelling,” ARM Community, Jan. 2019. [Online]. Available: https://community.arm.com/iot/b/internet-of-things/posts/five-steps-to-successful-threat-modelling?utm_source=google&utm_medium=cpc&utm_campaign=2019_ebg-security_mk01-1_na_na_bol&utm_term=five-steps-to-successful-threat-modelling&utm_content=blog&gclid=CjwKCAiAjrXxBRAPEiwAiM3DQnNOCgiKA2GuopshH_wYsalX1aa-Ekarmn5jvSSUlWZQj8GrTav_choCeroQAvD_BwE. [Accessed Apr.22, 2021].
  * [59] A. W. Atamli and A. Martin, “Threat-based security analysis for the Internet of Things,” in _2014 International Workshop on Secure Internet of Things_ , Sep. 2014, pp. 35–43.
  * [60] R. Hasan et al., “Toward a threat model for storage systems,” in _Proceedings of the 2005 ACM workshop on Storage security and survivability_ , Nov. 2005, pp. 94–102.
  * [61] M. Rushanan et al., “Sok: security and privacy in implantable medical devices and body area networks,” in _2014 IEEE Symposium on Security and Privacy_ , May 2014, pp. 524–539.
  * [62] E. Hutchins et al., “Intelligence-driven computer network defense informed by analysis of adversary campaigns and intrusion kill chains,” _Leading Issues in Information Warfare & Security Research_, vol. 1(1), p. 80, Apr. 2011\.
  * [63] N. Shevchenko et al., “Threat modeling: a summary of available methods,” Carnegie Mellon University Software Engineering Institute Pittsburgh United States, Tech. Rep., Jul. 2018.
  * [64] _Application of attack potential to smartcards_ , CCDB-2009-03-001 - Common Criteria, 2009.
  * [65] I. Verbauwhede, “Hardware security,” Bristol, UK, University of Bristol, CyBOK, 2019.
  * [66] O. Salem et al., “Sensor fault and patient anomaly detection and classification in medical wireless sensor networks,” in _2013 IEEE International Conference on Communications (ICC)_ , Jun. 2013, pp. 4373–4378.
  * [67] S. Aga and S. Narayanasamy, “InvisiMem: smart memory defenses for memory bus side channel,” _ACM SIGARCH Computer Architecture News_ , vol. 45(2), pp. 94–106, Jun. 2017.
  * [68] A. Syed and R. M. Lourde, “Hardware security threats to DSP applications in an IoT network,” in _2016 IEEE International Symposium on Nanoelectronic and Information Systems (iNIS)_ , Dec. 2016, pp. 62–66.
  * [69] M. Hasan and S. Mohan, “Protecting actuators in safety-critical IoT systems from control spoofing attacks,” in _Proceedings of the 2nd International ACM Workshop on Security and Privacy for the Internet-of-Things_ , Dec. 2019, pp. 8–14.
  * [70] S. Skorobogatov, “How microprobing can attack encrypted memory,” in _2017 Euromicro Conference on Digital System Design (DSD)_ , Aug. 2017, pp. 244–251.
  * [71] M. Brownfield, Y. Gupta, and N. Davis, “Wireless sensor network denial of sleep attack,” in _Proceedings from the Sixth Annual IEEE SMC Information Assurance Workshop_ , Jun. 2005, pp. 356–364.
  * [72] J. Zhang et al., “Power analysis attack on a lightweight block cipher GIFT,” in _Proceedings of the 9th International Conference on Computer Engineering and Networks_ , 2021, pp. 565–574.
  * [73] A. Lahmadi et al., “MitM attack detection in BLE networks using reconstruction and classification machine learning techniques,” in _MLCS 2020-2nd Workshop on Machine Learning for Cybersecurity_ , Sep. 2020, pp. 1–16.
  * [74] R. Fontana et al., “An innovative wireless endoscopic capsule with spherical shape,” _IEEE transactions on biomedical circuits and systems_ , vol. 11(1), pp. 143–152, Jun. 2016.
  * [75] Y. Jia et al., “A trimodal wireless implantable neural interface system-on-chip,” in _2020 International Solid-State Circuits Conference (ISSCC), Session 26 / Biomedical Innovations_ , Nov. 2020, pp. 414–415.
  * [76] I. Costanzo, D. Sen, and U. Guler, “An integrated readout circuit for a transcutaneous oxygen sensing wearable device,” in _2020 IEEE Custom Integrated Circuits Conference (CICC)_ , Mar. 2020, pp. 1–4.
  * [77] H. Ni, A. Chen, and N. Chen, “Some extensions on risk matrix approach,” _Safety Science_ , vol. 48(10), pp. 1269–1278, Dec. 2010.
  * [78] “Percept™ PC Neurostimulator,” Medtronic, 2020. [Online]. Available: https://www.medtronic.com/us-en/healthcare-professionals/products/neurological/deep-brain-stimulation-systems/percept-pc.html. [Accessed Apr.24, 2021].


validation loss with a patience of 10 and best weights are restored.
Validation split is 0.1.

In addition to the results presented in the main paper, we include results for
equality separators optimized with MSE and equality separators with bump
activations in their hidden layers with learnable $\sigma$. AUPRs of the test
dataset are reported in Table 14. From our results, equality separators with
RBF activations perform the best with a perfect AUPR during train and test
times, regardless whether they were optimized under LL or MSE.

Furthermore, as previously discussed, we observe a larger separation between
the positive class and the negative test class for these equality separators
with RBF activations optimized under LL. Looking at individual models, about
half of these equality separators achieved high separation between the normal
and anomaly class, where the difference between the mean output of normal data
and anomalies during testing is at least 0.30 (Figure 3(f)). In the
penultimate layer, this separation corresponds to normal data being a distance
within about 0.05 away from the hyperplane, while anomalies are a distance of
about 0.42 to 0.60 away. Even though the output of anomalies is still
relatively high (above 0.5), from the equality separation perspective, these
anomalies are more than 8 times further than normal data, so setting reliable
thresholds is more straightforward.

The difference between the means of the positive class and negative class
during test time was $0.27\pm 0.11$ for equality separators with RBF
activations optimized under LL, compared to $0.16\pm 0.11$ for those optimized
under MSE. In the penultimate layer of the NNs, the average difference
corresponds to a distance of the negative class from the hyperplane of 0.40
for LL optimization and 0.30 for MSE optimization (an output prediction of
$0.5$ corresponds to a distance of 0.59 away from the hyperplane). Since this
experiment had no overlap between the positive and negative class, LL
optimization was more appropriate, obtaining perfect AUPR like MSE
optimization but a greater separation between the positive and negative class.
A greater separation suggests one-class decision boundaries that are better.

The next best models achieving close to perfect AUPR are the equality
separators with bump activations and RBF separators with leaky ReLU or RBF
activations. NNs with bump activations in their hidden layers consistently
achieved mean AUPRs of above 0.90, suggesting the versality of the bump
activation as a semi-local activation function.

On a whole, it seems that a mix of globality and locality in activation
functions helps to increase one-class classification performance. While
equality separation with the local RBF activation was the most effective,
equality separators with bump activations (purely semi-local) and RBF
separators with leaky ReLU (local paired with global) also have competitive
AUPR. When using leaky ReLU in the hidden layers, increasing the locality of
the output activation increases performance, while the less global bump and
RBF activations also increase performance of halfspace separators. Meanwhile,
NNs with bump activations in their hidden layers consistently achieved over
0.90 mean AUPR. Our experiments corroborate with our intuition in Section 3.3
that more local mappings (such as the equality separator and its smooth
version, the bump activation) can implicitly encode a one-class classification
objective. The semi-local property of equality separators allows it to retain
locality for AD, while improving the separation between the normal and anomaly
class of NNs with only RBF activations.

Table 14: Test time AUPR for supervised AD using neural networks with 2 hidden
layers. For NNs, we modify the output layer (halfspace separation (HS),
equality separation (ES) and RBF separation (RS)) in the rows and activation
functions in the columns. For equality separators, we include information on
the loss optimized on (logistic loss LL or mean-squared error MSE). If
$\sigma$ is learnable in the bump activations in the hidden layers, we include
an added “s” suffix. (This table includes results from updated experiments
done in the main paper.)
NN$\backslash$Activation | Leaky ReLU | Bump | RBF
---|---|---|---
HS | 0.62$\pm$0.12 | 0.92$\pm$0.13 | 0.83$\pm$0.18
RS | 0.97$\pm$0.05 | 0.91$\pm$0.14 | 0.97$\pm$0.06
ES (LL) | 0.83$\pm$0.15 | 0.99$\pm$0.04 | 1.00$\pm$0.00
ES (MSE) | 0.77$\pm$0.18 | 0.98$\pm$0.09 | 1.00$\pm$0.00
ES-s (LL) | - | 0.98$\pm$0.05 | -
ES-s (MSE) | - | 0.95$\pm$0.11 | -

##### Ablation: 3 hidden layers

We also investigate NNs with 3 hidden layers, which is 1 more hidden layer
than our original experiments. We report test time AUPRs in Table 15. In
general, NNs with 3 hidden layers achieved worse performance than their
counterparts with 2 hidden layers. Coupled with train AUPRs of above 0.97, we
can see that NNs with 3 hidden layers generally overfit to the negative class
during training. Equality separator NNs with RBF activations with 3 hidden
layers optimized under LL maintained a perfect AUPR and achieved a separation
between the positive and negative class during test time of $0.22\pm 0.13$,
which has a mean that is only slightly lower than their counterpart with 2
hidden layers. Those optimized under MSE also maintained a close to perfect
AUPR with a separation of $0.16\pm 0.14$, which is similar to their
counterparts with 2 hidden layers.

In our experiments, NNs with bump activations in their hidden layers were
consistently resistant to severe overfitting while maintaining a relatively
good test performance of mean AUPR at least 0.89, with a mean test AUPR always
within 1 standard deviation of their counterparts with 2 hidden layers. On the
other hand, NNs with RBF activations in their hidden layers maintained a high
test AUPR except for halfspace separators. Meanwhile, NNs with leaky ReLU
generally maintained performance within 1 standard deviation. In general, our
results suggest that bump activations are more robust to overfitting while
being able to achieve good performance for anomaly detection. Future work to
more thoroughly investigate this overfitting phenomenon can be done to
understand how to further mitigate overfitting, especially without prior
exhaustive knowledge of all possible negative examples.

Table 15: Test time AUPR for supervised AD using neural networks with 3 hidden
layers. For NNs, we modify the output layer (halfspace separation (HS),
equality separation (ES) and RBF separation (RS)) in the rows and activation
functions in the columns. For equality separators, we include information on
the loss optimized on (logistic loss LL or mean-squared error MSE). If
$\sigma$ is learnable in the bump activations in the hidden layers, we include
an added “s” suffix.
NN$\backslash$Activation | Leaky ReLU | Bump | RBF
---|---|---|---
HS | 0.69$\pm$0.16 | 0.92$\pm$0.14 | 0.63$\pm$0.17
RS | 0.96$\pm$0.06 | 0.89$\pm$0.14 | 0.94$\pm$0.13
ES (LL) | 0.83$\pm$0.15 | 0.95$\pm$0.10 | 1.00$\pm$0.00
ES (MSE) | 0.84$\pm$0.16 | 0.90$\pm$0.13 | 0.99$\pm$0.03
ES-s (LL) | - | 0.98$\pm$0.04 | -
ES-s (MSE) | - | 0.90$\pm$0.16 | -

##### Learnable $\sigma$ in bump activations

Having a learnable $\sigma$ parameter in the hidden layers of the bump
activation generally do not affect AUPR significantly for equality separator
NNs optimized under LL or MSE. Moreover, a pattern emerges for both 2- and
3-hidden layered NNs – for individual NNs that performed well, the $\sigma$
value in the first hidden layer tends to increase while the $\sigma$ value in
the latter hidden layer tends to decrease, with the final value of $\sigma$
usually being higher at the first hidden layer than the latter layers after
convergence (Figure 8). A larger $\sigma$ in the earlier layer signifies
allowing more information through the layer because there is a larger margin,
while a lower $\sigma$ in the latter layers suggests a greater selectivity in
inputs for neuron activation. Intuitively, the first hidden layer increases
the dimensionality of the input for feature extraction (from 2D inputs to 5D
hidden layer), while the latter hidden layers perform variable selection based
on the increased dimensionality.

Such behavior of $\sigma$ is insightful to how gradient-based optimizers learn
equality separation (even in hidden layers and with fixed $\sigma$). Since
backpropagation through the latter hidden layers decreases the value of
$\sigma$ and decreases the receptive field of the bump activation, we can
infer that backpropagation is also likely to decrease the receptive field of
latter layers with fixed $\sigma$ in their bump activation. This behavior
corroborates with the observation in G.C. Vasconcelos (1995) that bump
activations lead to more closed decision regions. Such behavior is especially
useful in applications like anomaly detection, where equality separation and
bump activations seems to be inclined towards encoding the inductive bias of
one-class classification.

(a) Neural network with 2 hidden layers.

(b) Neural network with 3 hidden layers.

Figure 8: Example behaviour of $\sigma$ in the hidden layers of an equality
separator NN with bump activations and learnable $\sigma$ in its hidden
layers. The numbered suffix of the label of the curves refers to the index of
the hidden layer (starting from 1).

##### Model averaging for increased reliability

NNs with bump activations generally achieved high mean AUPR with some
variance. Particularly, equality separators with bump activations in their
hidden layers achieved the highest mean AUPR (close to perfect), for both 2-
and 3-hidden layered NNs. High mean AUPR with a high variance suggest that
individual NNs are able to achieve good separation of the positive and
negative classes during test time. Therefore, NNs may be slightly sensitive to
initializations. To mitigate potentially bad initializations, we can view
these NNs as weak learners for anomaly detection. Then, an ensemble model
should hopefully produce a better model for anomaly detection.

The simplest approach of averaging the predictions of the 20 models we trained
achieves a perfect test AUPR for both the 2-hidden layered NN (Figure 9(a))
and 3-hidden layered NN (Figure 9(b)). By averaging the prediction of the
numerous trained models, the bad initializations were cancelled out and the
ensemble model performs more reliably.

Other methods can be applied to ensemble the models, such as bootstrap
aggregation and boosting. Future work can also look into how to initialize and
optimize these NNs with bump activations well to mitigate bad initializations.
Other work to directly mitigate overfitting during training of equality
separators can be explored to increase the separation of normal data from
anomalies to move towards a separation like the shallow equality separator
with RBF kernel, such as what initializations pair well with NN optimization.

(a) 2 hidden layers.

(b) 3 hidden layers.

Figure 9: The heat map of the average of the output prediction of 20 equality
separator neural networks with bump activations and fixed $\sigma=0.5$.

###  F.4 Supervised anomaly detection with NSL-KDD dataset

We report additional details of experiments done in Section 4.2 for NSL-KDD
dataset. Since the dataset size is significantly larger than our synthetic
dataset in Section 4.1, we opt to test NNs with a homogenous type of
activation function (i.e. global, semi-local or local) and repeat experiments
3 times. We report more results in Table 16.

##### Unsupervised methods

For unsupervised AD methods, we test on the 2 options of dealing with labels
like before: feed all the data into the algorithm, or just feed in normal data
into the algorithm and remove the anomalies (because unsupervised methods
assume that most/all data fed in is normal). In the main paper, we report
results of only the former. Here, we report the former with a suffix “-A” and
the latter with a suffix “-N” in Table 16. We report the best result in bold,
but also report results one standard deviation away in bold to account for
variability across experiments with different seeds.

We see OCSVM significantly outperforming all the other methods on the unseen
anomalies for privilege escalation and remote access attacks. However, OCSVMs
are unable to capitalize label information, and it is expected that the
supervised SVM comfortably outperforms OCSVM as well. Once again, we notice a
trade-off of poorer detection seen anomalies with the former and poorer
detection of unseen anomalies for the latter. When all data is fed into an
unsupervised method, it is intuitive that the model may view anomalous data as
normal and misclassify them, leading to poorer detection of seen anomalies.
However, when only normal data is fed into the unsupervised method, it is
interesting to note that the resultant model is more conservative about the
unseen anomalies. Particularly, we notice IsoF obtaining the best AUPR for DoS
and probe attacks.

As an aside, we note that LOF is a distance-based algorithm, which is a likely
reason for its failure in this high-dimensional setting. Both LOF and methods
that require kernels (OCSVM) take a significantly longer time to train in
these high-dimensional settings too.

##### Binary classifiers

We details the hyperparameters for training our binary classifier NNs below.

  1. 1.

We train NNs for 500 epochs with an early stopping patience of 10 epochs, with
best weights restored. NNs were far from reaching the upper limit of 500
epochs.

  2. 2.

We used the Adam optimizer with learning rate of $3\times 10^{-4}$.

  3. 3.

We use 1 hidden layer with 60 neurons, about half of the input dimension.

  4. 4.

RBF separators and halfspace separators are trained with LL while equality
separators are trained with MSE. In this appendix, we experiment with both LL
and MSE and report results, noting that there is not much effect of the
respective classifiers strengths and weaknesses. We note that equality
separators trained with MSE also achieve similar performance to those trained
with LL.

  5. 5.

Some results were slightly sensitive, so we repeated experiments 5 times to
obtain a more reliable result.

  6. 6.

We use batch normalization before the activation function is applied (less
RBF, where the only option is to apply it after).

  7. 7.

Other hyperparameters are kept the same as our synthetic experiment (e.g.
leaky ReLU for global activations, Glorot uniform initializer for weights).

Our choice was partially guided by the observation that it is easy to overfit
the training data (i.e. training loss keeps decreasing while validation loss
does not). This observation corroborates with learning theory of how the
generalization error bound is looser with more expressive hypothesis classes.
In particular, we experimented with 1 hidden layer with 60 neurons, 1 hidden
layer with 200 neurons, 2 hidden layers with 60 neurons each and 2 hidden
layers with 200 neurons each.

##### NS

For NS, we maintain the same training regime but detail / modify the following
hyperparameters.

  1. 1.

The number of anomalies sampled was 10 times the size of the training dataset.

  2. 2.

We trained our equality separators with MSE. Since anomalies can be anywhere,
including close to the normal data, we use MSE to be more conservative about
wrong classifications, given the potential label noise when artificially
sampling anomalies.

##### SAD

For SAD, we pre-train an autoencoder on our training data, then use the
encoder as an intialization to train it for supervised AD as in Ruff et al.
(2020). Training details are as stated:

  1. 1.

We use encoder dimensions of 90-60 (i.e. a hidden layer of 90 neurons between
the latent layer of 60 neurons and the input/output). We employ this for all
architectures because we observe underfitting without the intermediate hidden
layer. We also validate performance by observing training error converge to 0
when the autoencoder only has 1 hidden layer, where the hidden layer has the
same number of neurons as the input/output.

  2. 2.

Depending on the type of separator we test, the autoencoder employs either
leaky ReLU (global), bump (semi-local) or RBF (local) activations
homogenously.

  3. 3.

Autoencoder and classification head is trained as per Ruff et al. (2020) with
hyperparameters as previously stated.

  4. 4.

L2 regularization for classification (Ruff et al., 2020) was tried with
$10^{-1}$ and $10^{-6}$ weights and found to not to be sensitive. We used a
weight of $10^{-1}$.

We note that we do not imply that RBFs are inappropriate for AD. In fact, its
locality property makes it well-suited. However, RBFs are tricky to train
(Goodfellow et al., 2015) and may require many neurons to work (Flake, 1993).
Equality separators achieve a good balance between its ability to form closed
decision regions and its capacity to learn efficiently.

Table 16: Test time AUPR for NSL-KDD. Negative sampling is denoted with a suffix (-NS). Overall AUPR against all attacks is also reported. Suffix for unsupervised AD methods determines if only normal data is used (-N) or all data is used (-A).  Model$\backslash$Attack | DoS | Probe | Privilege | Access | Overall
---|---|---|---|---|---
Random | 0.435 | 0.200 | 0.007 | 0.220 | 0.569
SVM | 0.950$\pm$0.000 | 0.626$\pm$0.000 | 0.003$\pm$0.000 | 0.123$\pm$0.000 | 0.894$\pm$0.000
OCSVM-N | 0.835$\pm$0.000 | 0.849$\pm$0.000 | 0.382$\pm$0.000 | 0.745$\pm$0.000 | 0.920$\pm$0.000
OCSVM-A | 0.779$\pm$0.000 | 0.827$\pm$0.000 | 0.405$\pm$0.000 | 0.760$\pm$0.000 | 0.897$\pm$0.000
IsoF-N | 0.964$\pm$0.006 | 0.960$\pm$0.003 | 0.039$\pm$0.007 | 0.438$\pm$0.015 | 0.957$\pm$0.002
IsoF-A | 0.765$\pm$0.073 | 0.850$\pm$0.066 | 0.089$\pm$0.044 | 0.392$\pm$0.029 | 0.865$\pm$0.031
LOF-N | 0.759$\pm$0.000 | 0.501$\pm$0.000 | 0.046$\pm$0.000 | 0.451$\pm$0.000 | 0.824$\pm$0.000
LOF-A | 0.495$\pm$0.000 | 0.567$\pm$0.000 | 0.039$\pm$0.000 | 0.455$\pm$0.000 | 0.718$\pm$0.000
HS | 0.944$\pm$0.016 | 0.739$\pm$0.018 | 0.016$\pm$0.006 | 0.180$\pm$0.033 | 0.877$\pm$0.010
ES (ours) | 0.970$\pm$0.006 | 0.726$\pm$0.093 | 0.047$\pm$0.017 | 0.446$\pm$0.124 | 0.939$\pm$0.014
RS | 0.356$\pm$0.002 | 0.213$\pm$0.002 | 0.010$\pm$0.000 | 0.228$\pm$0.002 | 0.406$\pm$0.001
HS, MSE | 0.921$\pm$0.015 | 0.682$\pm$0.017 | 0.010$\pm$0.006 | 0.133$\pm$0.008 | 0.846$\pm$0.008
ES, MSE (ours) | 0.974$\pm$0.001 | 0.717$\pm$0.107 | 0.045$\pm$0.014 | 0.510$\pm$0.113 | 0.941$\pm$0.014
RS, MSE | 0.356$\pm$0.000 | 0.215$\pm$0.000 | 0.010$\pm$0.000 | 0.231$\pm$0.000 | 0.406$\pm$0.000
HS-NS | 0.936$\pm$0.001 | 0.642$\pm$0.030 | 0.006$\pm$0.000 | 0.139$\pm$0.001 | 0.845$\pm$0.006
ES-NS (ours) | 0.945$\pm$0.009 | 0.659$\pm$0.013 | 0.023$\pm$0.011 | 0.206$\pm$0.013 | 0.881$\pm$0.007
RS-NS | 0.350$\pm$0.002 | 0.207$\pm$0.002 | 0.009$\pm$0.000 | 0.223$\pm$0.002 | 0.401$\pm$0.002
HS-SAD | 0.955$\pm$0.003 | 0.766$\pm$0.011 | 0.100$\pm$0.000 | 0.447$\pm$0.000 | 0.935$\pm$0.007
ES-SAD (ours) | 0.960$\pm$0.002 | 0.795$\pm$0.004 | 0.047$\pm$0.002 | 0.509$\pm$0.009 | 0.952$\pm$0.002
RS-SAD | 0.935$\pm$0.000 | 0.678$\pm$0.000 | 0.022$\pm$0.000 | 0.142$\pm$0.000 | 0.846$\pm$0.000

###  F.5 Supervised anomaly detection with thyroid dataset

We report additional details of experiments done in Section 4.2 for the
thyroid dataset. All details are the same as Section F.4 unless otherwise
stated.

The thyroid dataset is a medical dataset on proper functioning of the thyroid
(a gland in the neck), with 21 features and 3 classes, 1 of which is normal
while the anomalous classes are hyperfunction and subnormal. For Thyroid
dataset, we train on hyperfunction anomalies and test on both hyperfunction
and subnormal anomalies.

  1. 1.

By monitoring the validation loss (validation split of 0.15), we choose the
model that does not overfit.

  2. 2.

We choose 1 hidden layer with 200 neurons with batch normalization.

  3. 3.

We initialize $\sigma=7$ to increase the margin width so initializations are
not as sensitive. (We also note that $\sigma=5,10$ produce similar results.)

  4. 4.

We employ learning rate exponential decay with Adam optimizer under LL to
speed up convergence, with an initial learning rate of 0.01 and decay of 0.96.

  5. 5.

We train NNs for 500 epochs with early stopping patience of 50 epochs, with
best weights restored.

We report more detailed results in Table 17, which includes a logistic
regression baseline. We see that equality separation is the best at detecting
both seen and unseen anomalies.

Table 17: AUPR for thyroid by diagnosis (i.e. anomaly type). Diagnosis | Logistic Regression | HS | ES (ours) | RS
---|---|---|---|---
Hyperfunction (Seen) | 0.889$\pm$0.000 | 0.907$\pm$0.013 | 0.934$\pm$0.004 | 0.030$\pm$0.002
Subnormal (Unseen) | 0.340$\pm$0.001 | 0.459$\pm$0.040 | 0.596$\pm$0.007 | 0.082$\pm$0.002
Overall | 0.530$\pm$0.001 | 0.615$\pm$0.032 | 0.726$\pm$0.004 | 0.103$\pm$0.001

###  F.6 Supervised anomaly detection with MVTec dataset

We report additional details of experiments done in Section 4.2 for MVTec
defect dataset. All details are the same as Section F.4 unless otherwise
stated.

Each object has about 300 training samples, so we use (frozen) DINOv2 (Oquab
et al., 2023) to embed raw image data into features rather than training our
own computer vision model from scratch. DINOv2 is a foundation Vision
Transformer (ViT) model pre-trained on many images, so we assume that the
feature representations obtained from the model are useful, or at least more
useful than a model trained from scratch. Specifically, we use the
classification CLS token embedding, since it is not patch-dependent and is
widely used for downstream classification tasks.

There are far fewer data samples than the ambient dimensionality of the data,
so we opt for simple methods. Our halfspace, equality and RBF separators are
all output-layered (i.e. no hidden layers) and act as linear probes, except
the RBF separator which acts like SAD where DINOv2 plays the role of the
encoder (of a pre-trained autoencoder) in Ruff et al. (2020) to extract
meaningful features from the data. Hence, the halfspace separator acts as our
baseline. In high dimensions, sampling to cover the whole space is much harder
and computationally heavier, so we exclude negative sampling (NS).

Each anomaly class has very few anomalies (usually around 10), so we use all
anomalies from 1 anomaly type during training (we pick the first type of
anomaly in alphabetical order). In other words, we do not test on seen
anomalies and only test on unseen anomalies. As the best proxy (or at least an
upper bound) for performance on seen anomalies, we monitor the training loss
and AUPR as well. For instance, we observe that RBF separators do not converge
and have low AUPR, which suggests that test AUPR on seen anomalies is likely
to be worse. Meanwhile, the two other models (halfspace and equality
separators) had converging losses and good train AUPR. Out of these 2 models,
equality separators had better test AUPR (i.e. better separation between
normal and unseen anomalies) on a whole.