DNN-Alias: Deep Neural Network Protection Against Side-Channel Attacks
  via Layer Balancing
Extracting the architecture of layers of a given deep neural network (DNN)
through hardware-based side channels allows adversaries to steal its
intellectual property and even launch powerful adversarial attacks on the
target system. In this work, we propose DNN-Alias, an obfuscation method for
DNNs that forces all the layers in a given network to have similar execution
traces, preventing attack models from differentiating between the layers.
Towards this, DNN-Alias performs various layer-obfuscation operations, e.g.,
layer branching, layer deepening, etc, to alter the run-time traces while
maintaining the functionality. DNN-Alias deploys an evolutionary algorithm to
find the best combination of obfuscation operations in terms of maximizing the
security level while maintaining a user-provided latency overhead budget. We
demonstrate the effectiveness of our DNN-Alias technique by obfuscating the
architecture of 700 randomly generated and obfuscated DNNs running on multiple
Nvidia RTX 2080 TI GPU-based machines. Our experiments show that
state-of-the-art side-channel architecture stealing attacks cannot extract the
original DNN accurately. Moreover, we obfuscate the architecture of various
DNNs, such as the VGG-11, VGG-13, ResNet-20, and ResNet-32 networks. Training
the DNNs using the standard CIFAR10 dataset, we show that our DNN-Alias
maintains the functionality of the original DNNs by preserving the original
inference accuracy. Further, the experiments highlight that adversarial attack
on obfuscated DNNs is unsuccessful.
Mahya Morid Ahmadi, Lilas Alrahis, Ozgur Sinanoglu, Muhammad Shafique
# DNN-Alias: Deep Neural Network Protection Against Side-Channel Attacks via Layer Balancing

###### Abstract

Extracting the architecture of layers of a given deep neural network (DNN) through hardware-based side channels allows adversaries to steal its intellectual property and even launch powerful adversarial attacks on the target system. In this work, we propose _DNN-Alias_, an obfuscation method for DNNs that forces all the layers in a given network to have similar execution traces, preventing attack models from differentiating between the layers. Towards this, DNN-Alias performs various layer-obfuscation operations, e.g., layer branching, layer decepening, etc, to alter the run-time traces while maintaining the functionality. DNN-Alias deploys an evolutionary algorithm to find the best combination of obfuscation operations in terms of maximizing the security level while maintaining a user-provided latency overhead budget.

We demonstrate the effectiveness of our DNN-Alias technique by obfuscating the architecture of 700 randomly generated and obfuscated DNNs running on multiple Nvidia RTX 2080 TI GPU-based machines. Our experiments show that state-of-the-art side-channel architecture stealing attacks cannot extract the original DNN accurately. Moreover, we obfuscate the architecture of various DNNs, such as the VGG-11, VGG-13, ResNet-20, and ResNet-32 networks. Training the DNNs using the standard CIFAR10 dataset, we show that our DNN-Alias maintains the functionality of the original DNNs by preserving the original inference accuracy. Further, the experiments highlight that adversarial attack on obfuscated DNNs is unsuccessful.

## I Introduction

Deep neural networks (DNNs) have experienced rapid advancements in the past decade, leading to their application to many areas of human endeavor and various fields of science [1, 2]. The deployment of DNNs in mission-critical applications, such as healthcare systems [3] and anomaly detection in cyber-physical systems [4], raises concerns about the safety and security of these networks. Moreover, building and training DNNs require expert knowledge and costly resources. Thus, DNNs for a certain application are considered sensitive and expensive intellectual property (IP) that require protection from malicious users and/or market competitors.

To facilitate the application of DNNs, model developers offer machine learning as a service (MLaaS), which includes machine learning (ML) models and services running on the cloud or edge devices [5]. In the MLaaS business model, the end-user receives input/output access to the DNN, i.e., _black-box access_. An untrusted user would want to extract the IP of the underlying DNN, for launching _white-box adversarial attacks_ on the target system and/or for stealing the IP without incurring high research and development costs.1 Note, it is difficult to launch such attacks with only back-box access to the system, e.g., it takes 30 GPU days to launch an adversarial attack on Lenet+ (7 layers) using input/output queries [9].

Footnote 1: An adversarial attack performs subtle perturbations to the input samples of an ML model, causing the model to predict incorrect outputs [6, 7]. A white-box attack model assumes access to the inputs, architecture, and internal details, e.g., weights. Conversely, in the black-box model, the attacker lacks access to these details. A gray-box attack trains a substitute model to generate adversarial samples and attack the target model [8].

Additional information about the target DNN could be extracted through _hardware-based side-channels_, such as the DNN architecture (i.e., the number, type, dimension, and connectivity of layers), enabling advanced adversarial attacks. For example, _DeepSniffer_[10] is a DNN side-channel-based architecture stealing (SCAS) attack that learns the correlation between the architecture hints (such as volumes of memory writes/reads) and the DNN architecture.2 DeepSniffer showed that including the architecture information increases the success rate of an adversarial attack by \(\approx 3\times\). Fig. 1 shows an example of such SCAS attack flow.

Footnote 2: SCAS attacks can be physical (edge) or remote (cloud) [11].

To demonstrate the potent nature of such SCAS attacks, we plot the run-time traces of the AlexNet DNN trained on the CIFAR10 dataset in terms of computation latency (cycles) and memory access time (read and write) per layer in Fig 2. It can be observed that each type of layer has a unique execution signature. As a result, SCAS attacks learn the execution pattern of the layer functions and extract the layer sequence.

Researchers have studied SCAS attacks based on various side-channels, e.g., power [12]. We focus on timing (execution time and memory access) side-channels, which are more stealthy, as they exploit an operational part of system (system profilers) for measuring and do not require extra equipment.

Researchers have developed several protection techniques

Fig. 1: Hardware-based side-channel leakage facilitates DNN architecture stealing, leading to white-grey-box adversarial attacks.
to thwart SCAS attacks. Some _hiding_ techniques decrease the signal-to-noise ratio in side-channel traces, e.g., via dummy memory operations [13]. Other methods necessitate hardware modifications to encrypt memory and other side-channel leakage [14]. More recently, DNN obfuscation has been proposed to alter the run-time traces of a given DNN while preserving its functionality, thwarting SCAS attacks [15, 16]. Nevertheless, these methods suffer at least from one of the drawbacks discussed next and summarized in Table I.

### _State-of-the-Art (SOTA) and their Limitations_

**Overhead:** Oblivious random access memory (ORAM) schemes encrypt and shuffle the memory read/writes, reducing memory leakage [18]. However, ORAM-protected designs suffer from high overhead, e.g., \(\approx 10\times\) latency cost [20].

**Infectiveness:** Introducing noise to the execution traces thwarts statistical SCAS attacks, but fails to mitigate ML-based SCAS attacks.3 Such ML-based attacks are trained to be resilient to noisy data. For example, the success rate of the ML-based _DeepSniffer_[10] attack remains the same even with 30% of amplitude noise.

Footnote 3: In first, the attacker applies statistical methods, i.e., correlation analysis, to distinguish the correct secret value among the hypotheses [12], while in second, the attacker trains an ML model to classify the traces.

**Limited and Hardware-Specific Security:** Current DNN obfuscation methods offer limited security. It has been shown that advanced attacks, such as _NeuroUnlock_[17], can learn the obfuscation procedure and automatically revert it, thereby recovering the original DNN architecture. Further, existing obfuscation techniques, such as _NeuroObfuscator_[15], depend on extensive hardware profiling. This profiling step makes the defense mechanism dependent on the underlying hardware.

### _Key Research Challenges Targeted in this Work_

The above discussion shows that there is still a gap in designing secure DNN architectures. _Developing an efficient and cost-effective defense mechanism imposes the following important research questions and challenges._

1. **What makes SCAS attacks successful?** To design secure DNNs, we need to identify the conditions that make the SCAS attacks successful and eliminate them during the development stage of the DNN.
2. **Generic security metric.** Defense mechanisms that focus solely on reducing an attack-specific metric, such as the layer error rate (LER),4 cannot mitigate further attack vectors. Thus, devising a generic security metric is required to evaluate the security of DNNs at design time. Footnote 4: The editing distance between extracted and original layer sequence.
3. **Performance overhead.** A generic and adaptive defense mechanism is required, which can be tailored per target DNN, hardware implementation, and overhead.

### _Our Novel Concept and Contributions_

To address the above challenges, we propose _DNN-Alias_, a DNN obfuscation methodology to protect the architecture of DNNs against static and ML-based SCAS attacks. We argue that SCAS attacks are successful because each layer has a unique run-time trace signature (see Fig. 2). Therefore, we demonstrate that if the signatures of the different layers overlap, it will be difficult for any SCAS attack to differentiate between them. Deterministic DNN obfuscation alters the run-time traces of the layers. However, it does not guarantee overlapping signatures. Our proposed DNN-Alias employs, for the first time, a generic security metric, which measures the overlap between layer signatures by computing the standard deviation. DNN-Alias performs various DNN obfuscation operations to minimize the standard deviation, i.e., resulting in a more secure DNN architecture. Note that the security objective does not depend on any specific attack output but rather on the features of the DNN itself. Furthermore, DNN-Alias does not profile the underlying hardware. Hence, DNN-Alias is a generic defense methodology that is applicable to any DNN running on any hardware. Our novel contributions are summarized in Fig. 3 and discussed below.

1. **General solution to measure the diversity of DNN layers (Sec. III-C):** DNN-Alias defines a novel security metric based on the distribution of run-time parameters to measure the overlap between layer signatures.

Fig. 3: Our contributions presented in this work are in blue box.

Fig. 2: The run-time trace of AlexNet. The unique pattern of the memory access bytes and computational cycles per each kernel allows SCAS attacks to learn these features and extract the target DNN architecture successfully.
Specifically, DNN-Alias calculates the standard deviation of features in the run-time trace and reduces it. By utilizing this metric, DNN-Alias becomes independent of the outcome of any particular attack.
2. **Balancing the execution trace (Sec. III-B):** DNN-Alias presents a novel DNN obfuscation technique that performs layer balancing, limiting information leakage.
3. **Efficient DNN obfuscation (Sec. III-D):** DNN-Alias employs a genetic algorithm to find an effective combination of obfuscation operations. The reward function of this algorithm minimizes the standard deviation in run-time trace while maintaining cost budget.

**Key Results:** We have comprehensively evaluated the efficacy of DNN-Alias with a broad set of random and standard DNNs on image classification, running on the Nvidia RTX 2080 TI GPU. We demonstrate that DNN-Alias increases the LER (between the original and obfuscated DNN) by \(2.5\times\) compared to SOTA techniques, resulting in higher obfuscation.

We evaluate the security of DNN-Alias by launching; (i) an ML-based SCAS attack and (ii) NeuroUnlock on the obfuscated DNNs. We measure the difference between the architecture of the original DNN and the recovered DNN by the attacks. The LER obtained on the DNN-Alias networks is 2 on average (an LER higher than 1 is considered secure). Further, NeuroUnlock fails to de-obfuscate the networks, reporting an LER of 1.1 on DNN-Alias networks.

Further, DNN-Alias preserves the training accuracy while protecting the DNN against gray-box adversarial attacks.

## II Background

In this section, we provide the necessary background required to protect DNNs against SCAS attacks.

### _Side-channel-based Architecture Stealing (SCAS) Attacks_

Adversaries can compromise the security of a DNN system by uncovering its confidential model components, such as its architecture and parameters. Extracting an exact copy of the DNN is challenging in a black-box setup, where access to the victim model is limited [21]. However, physical access to the DNN's hardware platform can lead to the exposure of confidential information through side-channel attacks, such as power analysis or timing analysis [22] (see 1 in Fig. 4).

Through a SCAS attack, the adversary builds a substitute model using the extracted information and trains it by querying the victim DNN or using a publicly available labeled dataset (see 1 and 1 in Fig. 4). The attacker can then use the substitute model to launch adversarial attacks against the original DNN system (see 1 in Fig. 4).

In this study, we focus on attacks that exploit memory, cache, and timing-based information leaks to reveal the architecture of DNNs running on GPU devices. For example, the DeepSniffer attack [10], depicted in Fig. 5, employs a long-short-term-memory (LSTM) model [24] to deduce the layer arrangement of a targeted DNN based on its runtime trace.

The run-time trace is a time-series collection of various characteristics, such as execution time and dynamic random access memory (DRAM) access time. The training process for the LSTM model involves profiling randomly generated DNNs on the target GPU. The layer sequence of each generated DNN is encoded as a vector and its runtime trace is extracted. Subsequently, a runtime profile dataset is constructed based on kernel-aware architectural hint, and used to train the LSTM. Attacker utilizes this LSTM as a predictor model, to detect the layer sequence of the DNN from run-time traces. Once the attacker obtains the layer sequence from predictions and construct the model, the dimensions of each layer are determined based on the predicted operation and its position in the time-step and final model is extracted.

To evaluate the accuracy of the SCAS attack, the LER metric has been adopted in literature to measure the difference between the original and extracted DNN layer sequence, which we also use in our analysis. LER is calculated as follows:

\[LER=\frac{ED(L,L^{*})}{|L^{*}|} \tag{1}\]

where \(L\) represents the predicted layer sequence, \(L^{*}\) represents the ground-truth, and \(|.|\) denotes the length of a sequence. \(ED(p,q)\) denotes the edit distance between the \(p\) and \(q\) sequences, i.e., the minimum number of insertions, substitutions and deletions required to change \(p\) into \(q\) (also referred to as the _Levenshtein_ distance [25]).

## III Our Proposed Defense Mechanism: DNN-Alias

We propose a DNN obfuscation method, DNN-Alias, to thwart SCAS attacks. In this section, we explain the steps of DNN-Alias in detail and summarize them in Fig. 6. Further, we discuss the attack model and its assumptions in Sec. III-A.

DNN-Alias takes an unprotected DNN as input and modifies its architecture by applying layer obfuscation techniques (Sec. III-B). The goal is to reduce the diversity in the run-time behavior of layers in the given DNN. To measure the diversity of the DNN's run-time behavior, DNN-Alias automatically

Fig. 4: Side-channel-based architecture stealing (SCAS) attack.

Fig. 5: Flow of the DeepSniffer attack [23].
analyzes its profile during one inference execution (Sec. III-C). DNN-Alias employs a genetic algorithm to guide the obfuscation and balance the DNN layers, taking into account the overhead budget (Sec. III-D).

### _Threat Model and Assumptions_

Consistent with most recent related works [26], we assume that the adversary has no prior knowledge of the victim DNN architecture, parameters, training algorithms, or hyper-parameters. We focus on edge security, in which the attacker has (i) system privilege access to the GPU platform encapsulating the victim DNN, (ii) the inputs and outputs (labels) of the DNN, and (iii) a publicly available training dataset. _We show that even with such a powerful threat model, attackers cannot steal the architecture of the DNNs obfuscated via our proposed DNN-Alias._

### _DNN Layer Sequence Obfuscation Operations_

DNN-Alias uses function-preserving obfuscation operations to protect the architecture of DNNs. DNN-Alias carefully apply one (or more) operation to each layer of the original model, making the target DNN more difficult to reverse engineer or attack (Step 1 in Fig. 6).

Please note that DNN obfuscation operations have been used before to protect the DNN architecture [27, 15]. DNN-Alias applies the same obfuscation knobs to change the run-time profile of each layer function and hide the architecture of original DNN. However, existing solutions either randomly obfuscate the layers or focus on a specific SCAS attack, resulting in weak protection. DNN-Alias guides the obfuscation differently, thwarting all SCAS attacks. Next, we explain the obfuscation operations used by DNN-Alias.

Let the matrix \(\mathbf{W}_{k1,k2,c,j}^{(i)}\) represent the \(i^{th}\) convolutional layer to be modified. \(k1\) and \(k2\) represent the height and width of the convolution kernel, respectively, while \(c\) and \(j\) denote the input and output channel size, respectively. \(\mathbf{X}^{(i)}\) and \(\sigma(\cdot)\) denote the input of the layer and the activation function (e.g., _ReLU_), respectively. Fig. 7 illustrates the original operator.

**Layer Branching.** dividing a single layer operator into smaller, partial operators, as demonstrated in Fig. 7. For example, a 2-D convolution layer (_Conv2D_) \(\mathbf{W}_{k_{1},k_{2},c,j}^{(i)}\) can be separated into two partial convolutions, as follows.

\[\begin{split}&\mathbf{U}_{k_{1},k_{2},c,j/2}^{(i)}=\mathbf{W}_{k_{1},k_{2},c,m}^{(i)}\quad m\in\left[0,\lfloor\frac{j}{2}\rfloor\right),\\ &\mathbf{V}_{k_{1},k_{2},c,j/2}^{(i)}=\mathbf{W}_{k_{1},k_{2},c,m}^{(i)} \quad m\in\left[\lfloor\frac{j}{2}\rfloor,j\right)\end{split} \tag{2}\]

The final output is obtained by combining the two partial results.

\[\mathbf{U}_{k_{1},k_{2},c,j/2}^{(i)}*\mathbf{X}^{(i)}||\mathbf{V}_{k_{1},k_{2},c,j/2}^{(i) }*\mathbf{X}^{(i)} \tag{3}\]

The splitting can also be performed in the input channel dimension. The final output in this case is the addition of the two, as follows, where \(\mathbf{X}^{(i)}\) is sliced into \(\mathbf{A}^{(i)}\) and \(\mathbf{B}^{(i)}\).

\[\mathbf{U}_{k_{1},k_{2},c/2,j}^{(i)}*\mathbf{A}^{(i)}+\mathbf{V}_{k_{1},k_{2},c/2,j}^{(i) }*\mathbf{B}^{(i)} \tag{4}\]

**Layer Skipping.** An additional Conv2D layer \(\mathbf{U}_{k_{1},k_{2},j,j}^{(i+1)}\) with all its parameters set to \(0\) is inserted to retain the original functionality. An illustration of this is shown in Fig. 7. The Conv2D layer can be expressed as follows, with \(\sigma\left(\mathbf{X}^{(i+1)}\right)\) representing the activation output of \(i^{th}\) original layer.

\[\sigma\left(\mathbf{X}^{(i+1)}\right)+\sigma\left(\mathbf{U}^{(i+1)}*\mathbf{X}^{(i+1)} \right)=\sigma\left(\mathbf{X}^{(i+1)}\right) \tag{5}\]

**Layer Deepening.** adds a new computational layer to the sequence. The new layer is inserted after the activation of the current layer and before the _batch normalization_ (BN) step as shown in Fig. 7. If the previous layer is linear, the newly added layer \(\mathbf{U}^{(i+1)}\) is initialized as an identity matrix \(\mathbf{I}\) to preserve the function of the model. Otherwise, \(\mathbf{U}_{k_{1},k_{2},j,j}^{(i+1)}\) can be generalized as:

\[\mathbf{U}_{a,b,c,d}^{(i+1)}=\left\{\begin{array}{cc}1&a=\frac{k_{1}+1}{2} \wedge b=\frac{k_{2}+1}{2}\wedge c=d\\ 0&\mathrm{otherwise}\end{array}\right. \tag{6}\]

Layer deepening is effective as long as the activation function satisfies the following condition, like the ReLU function.

\[\forall x:\sigma(x)=\sigma\left(\mathbf{I}*\sigma(x)\right) \tag{7}\]

Post obfuscation, the computation graph is extracted and fed to the TVM\({}^{\text{\textregistered}}\) compiler [28]. The compiler performs optimizations at the graph and operator levels, generating low-level optimized code for GPU execution.5

Footnote 5: DNN obfuscation can be categorized into sequence and dimension obfuscation. We focus on sequence obfuscation since the sequence identification stage is the most fundamental step in SCAS attacks.

### _Measuring the difference of the layers_

DNN-Alias guides the obfuscation algorithm to assemble a configuration of obfuscation knobs on a given DNN (Sec. III-D). For each obfuscated DNN, DNN-Alias analyzes its run-time execution trace in inference mode and computes the difference of the levels in the trace using a generic

Fig. 6: Proposed DNN-Alias methodology for DNN architecture protection.

Fig. 7: Visualization of the employed obfuscation operations [27, 15].
measurement technique (See 1 in Fig. 6). In this technique, the run-time traces are collected and analyzed online and target hardware is not required to be profiled in advance.

DNN-Alias measures the difference of the levels in a run-time trace using the standard deviation (\(St.D\)) of values in a kernel per each feature in the run-time profile of the DNN.6

Footnote 6: A standard deviation is a measure of how dispersed the data is in relation to the mean. A low standard deviation means data are clustered around the mean, and a high standard deviation indicates data are more spread out.

\[St.D=\sqrt{\frac{1}{N-1}\sum_{i=1}^{N}(x_{i}-\overline{x})^{2}} \tag{8}\]

\(N\) is number of kernels, \(x_{i}\) is the value in run-time trace and \(\overline{x}\) is the mean of the values. First, the original DNN is configured by incorporating the obfuscation operations. Then, the obfuscated DNN is executed on the hardware platform and \(St.D\) of the above features is calculated. DNN-Alias uses a genetic algorithm to find the best configuration of the obfuscation operations, by decreasing the \(St.D\) value so that the run-time traces for all layers are similar. Decreasing the \(St.D\) (difference) enables layer balancing and hides the function of the kernel from the attacker's predictors.

In Fig. 8, we present an example of DNN-Alias obfuscation by comparing the execution trace of the unprotected (black) and obfuscated (blue) ResNet-20. The execution of each layer in the network involves reading weights and inputs from memory through a kernel process, executing the layer's function, and writing the output back to memory. The combination of these three values for each trace per kernel creates a unique pattern, which reveals information to SCAS attacks and allow them to predict the function of the layer. To counter this, DNN-Alias reduces the difference between the traces and forces all layers to have similar run-time traces.

```
1:procedureOfuscation(\(DNN\), \(budget\), \(size_{p}\), \(generations\))
2:\(population\leftarrow\) generate \(size_{p}\) variations of \(DNN\)
3:for\(generation_{index}\) in \(0...generations\)do
4:\(list_{rewards}\leftarrow\) empty list
5:for\(population_{element}\) in \(population\)do
6:\(p
**Random DNN obfuscation.** A random selection process is utilized to determine the insertion and configuration of obfuscation operations within each layer of the specified DNN. For each obfuscation operation, a binary random decision is made with a \(50\%\) likelihood to determine if the operation will be utilized or disregarded. As a result, a layer may incur the application of 1 and 2 (each \(37.5\%\) probability), 3 or none (each \(12.5\%\) probability) obfuscation operations.

**Fitness function.** Then, we evaluate each member of the population using the fitness score, which is determined by the \(St.D\) and a scaled value of the cost budget, as defined below.

\[Fitness=\quad\sum_{i=1}^{N}St.D_{i}(S).\bigg{[}\bigg{(}\frac{T-(1+B)T^{*}}{T^ {*}}\bigg{)}^{2}\bigg{]} \tag{10}\]

**Crossover and mutation.** For the mating process, we rank the population based on their fitness scores and select the top half of individuals with the best scores as parents and add them to the next candidate pool. Also, parents are brought to the crossover process, where they combine to form an equal number of offspring that are added to the pool of candidates. The offspring are built from the crossover of the parents' obfuscation list, then mutated by adding Gaussian noise. These methods are known as 1-point crossover and Gaussian mutation in literature. As shown in Fig. 9, the trend of the fitness score gradually decreases with each pool, and the best candidates are carried forward to the next generation.

**Final solution.** The mutation process continues until the fitness score converges and stabilizes, which we found to occur after 20 generations in our case.

## IV Experimental Setup

In this section, we present our experimental setup in evaluating the effectiveness and security of DNN-Alias.

**Hardware.** For our experiments, we utilized the Nvidia RTX 2080 Ti GPU as our experimental platform. However, our proposed method is generic and applicable to other GPUs and hardware platforms. We employ Nsight(r) Compute [29] for profiling the GPU and launching the ML-based SCAS attack, which requires privileged access to the performance counters. A dataset of randomly generated DNNs was generated to profile them on the GPU and train the attack model predictors. Next, we explain how we create the dataset of random DNNs.

**Obfuscation Algorithm.** We implement DNN-Alias using the _PyTorch_ deep learning framework [30]. The DNNs (original and obfuscated) are described as Python model files. The genetic algorithm is implemented by using _pymoo_[31].

**Random DNN Creation.** To train the predictors of SCAS attack, we use the following method for generating 5,000 different DNNs for image classification, using the CIFAR-10 dataset as a reference. The number of Conv2D layers in each DNN is randomly selected from the range [4, 12], and the number of fully connected (FC) layers is randomly selected from the range [1, 4]. The output channel sizes of Conv2D layers and the dimensions of FC layers are also randomly chosen from a range of predefined values. Some Conv2D layers are randomly replaced with blocks from the ResNet and MobileNet networks, and some Conv2D layers are changed to pooling layers. Batch normalization (BN) layers are added after each Conv2D and FC layer. All DNNs have 3 input channels, width and height of 32, and 10 output classes. Using this technique, we also generate 700 random DNNs to analyze the performance of DNN-Alias in obfuscating random DNNs.

**SCAS-based Adversarial Attack.** In an adversarial attack scenario, the attacker manipulates the output of a DNN by adding subtle, almost imperceptible alterations to the input images. The objective of the attack is to find the smallest possible changes in the input that can cause the DNN to produce incorrect output, either arbitrarily (in case of untargeted attack) or as pre-determined (in case of targeted attack). To launch an adversarial attack on a DNN that operates as a gray box, the attacker often develops a substitute model by examining the input and output of the victim DNN. With the help of the SCAS attack, the adversary has access to the details of target DNN with high accuracy to build the substitute model. Then, adversarial samples are created using the white-box substitution technique. Finally, these adversarial samples are utilized to disrupt the workings of the target DNN.

In summary, Fig. 10 depicts the transfer-based adversarial attack flow, which includes the following steps:

#### Iv-1 Substitute Models

In this step, we train substitute models to closely mimic the target model's behavior. For black-box adversarial attacks, the substitute models are selected from publicly available DNN families. In SCAS-based adversarial attacks, the substitute model is obtained from runtime traces. We compare the success rate of adversarial attacks in both scenarios.

#### Iv-2 Adversarial Sample Generation

The most advanced methods utilize an ensemble approach to increase the likelihood of a successful attack, based on the idea that if an adversarial image is able to fool multiple models, it is more likely to have a similar effect on the black-box model. We

Fig. 10: Adversarial attack flow on the extracted DNN architecture.

Fig. 9: Fitness score trend of DNN-Alias on ResNet-20 with a population of 16. Here, we have 320 elements and 20 mutations.
follow the same procedure to produce adversarial images for our target DNNs.

#### Iv-A3 Deployment of Adversarial Samples

We pass the generated adversarial examples as input data to launch an attack on the gray-box DNN.

## V Security Analysis and Overhead Results

In this section, we present the security evaluation and performance analysis of the DNN-Alias compared to SOTA.

### _Effectiveness of DNN-Alias_

In Fig. 12, we show the obfuscation and SCAS attack evaluation procedure. The original DNN is first obfuscated by DNN-Alias (step ). Next, the runtime traces are collected to extract the DNN architecture via an ML-based SCAS attack (step ). By comparing the LER between the original and extracted DNNs, we can analyze the effectiveness of the obfuscation method (step ). The best case scenario for the SCAS attack is to obtain an LER close to \(0\).

#### V-A1 Effectiveness on Random DNNs

We obfuscate \(700\) randomly generated DNNs (explained in Sec. IV) using DNN-Alias, launch the SCAS attack, and measure the security in terms of the LER (Extracted_Obf, Original). These results are shown in Fig. 11. The cost budget considered in this experiment is set to \(0.2\). The minimum LER value observed in this experiment is \(0.3\) and the maximum is \(3.5\). The FORECAST calculation of LER (predicts a future value by using linear regression) shows \(1.2\). The average of LER for SOTA techniques, is shown in Fig. 11, where LER for NeurObfuscator [15] is \(0.62\) and for ReDLock [17] is \(0.73\). Therefore, _DNN-Alias obfuscation is \(\approx 2\times\) more resilient against SCAS attacks compared to SOTA._

#### V-A2 Effectiveness on Publicly Available DNNs

Further, we analyzed the effectiveness of DNN-Alias on a set of real DNNs as a case study. The results are shown in Fig. 13.

First, we launch the ML-based SCAS attack on the original DNNs, i.e., without any obfuscation. The red bar for LER (Extracted_org, Original) shows the ML-based SCAS predictor errors with an average of \(0.05\), indicating that the original DNNs are completely vulnerable to SCAS attacks. Our goal is to increase this LER value over \(1\).

Next, each DNN was obfuscated using DNN-Alias for two cost budgets (\(0.2\) and \(0.6\)). We launch the same ML-bases SCAS attack on the obfuscated DNNs.

The blue bar LER (Extracted_obf, Original) represents the difference between the extracted DNN by the SCAS attack and the original DNN, which averages \(1.8\) for DNN-Alias. The increase of the LER from an average of \(0.05\) to \(1.8\) demonstrates that DNN-Alias is highly effective in protecting the DNN against SCAS.

#### V-A3 Obfuscation Overhead and Cost Budget

In all cases presented in Fig. 13, we can see that increasing the cost budget (from \(0.2\) to \(0.6\)) increases the LER (from an average of \(1.7\) to \(1.96\)), i.e., leads to stronger obfuscation.

The gray bar LER (Obfuscated, Original) in Fig. 13 represents the difference between the original DNN and the obfuscated DNN. The LER, in this case, has an average of \(0.1\), demonstrating that _DNN-Alias effectively thwarts SCAS attacks through minor changes in the original DNN_, avoiding high overhead costs.

#### V-A4 NeuroUnlock Attack on DNN-Alias

We evaluated the SOTA NeuroUnlock attack [17] on DNN-Alias and NeurObfuscator [15]. NeuroUnlock attempts to reverse the obfuscation of the extracted DNN from the SCAS attack using sophisticated ML-based models. The green bar labeled "LER (Recovered, Original)" in Fig.13 shows the difference between the original DNN and the recovered DNN using NeuroUnlock when DNN-Alias is in place. With an average LER of \(0.9\), the results indicate that NeuroUnlock failed to accurately recover the original DNN. In comparison, recovering the obfuscated DNN using NeuroUnlock with NeurObfuscator in place resulted in an average LER of \(0.31\). This suggests that _DNN-Alias is \(\approx 3\times\) more robust against de-obfuscation techniques than the current SOTA methods._

#### V-A5 Comparison with SOTA

We compare the effectiveness of DNN-Alias to NeurObfuscator [15] considering the target real DNNs. We obfuscate the DNNs with \(0.2\) and \(0.6\) latency budgets using DNN-Alias and NeurObfuscator (the code is open-sourced). We apply the SCAS attack on the obfuscated DNNs and compare each extracted DNN to the original DNN.

The results in Fig. 14 show that the LER of obfuscated DNNs using DNN-Alias is \(2.5\times\) (on average) more than NeurObfuscator. _In summary, DNN-Alias is more effective in hiding the layer sequence of DNNs compared to SOTA_.

### _Performance Analysis_

In this section, we examine the effect of DNN-Alias on the DNN training and the overhead involved in its design. Last but not least, we study the success rate of adversarial attacks against DNN-Alias networks.

Fig. 11: LER of random DNNs obfuscated by DNN-Alias and SOTA techniques.

Fig. 12: Measuring the accuracy of the SCAS attack, when the DNN is protected by obfuscation methods.
#### V-B1 Training Performance

To assess the effect of DNN-Alias on the functionality of the DNN, we compare the validation accuracy of the original and obfuscated DNNs. Further, we train the DNNs recovered by the SCAS attack and by NeuroUnlock attack to see if the high LER values map to loss in DNN performance. Fig. 15 shows the validation accuracy of the VGG-11 DNN over \(30\) epochs of training on the CIFAR-10 dataset [32].

The results demonstrate that DNN-Alias (blue line) maintains the functionality of the DNN and does not impact the training performance. However, the recovered DNN by the SCAS attack (green line) does not work and simply does not converge, with a \(75\%\) drop in performance. Further, even if NeuroUnlock was launched after the SCAS attack to revert the obfuscation, the recovered model (black line) still shows a lower validation accuracy, with a drop of \(10\%\), and converges about \(8\) epochs later. Therefore, _DNN-Alias forces the SCAS attack to recover a DNN with worse performance compared to the original DNN_.

#### V-B2 Adversarial Attack

We launch the SCAS-based adversarial attacks (discussed in Sec. IV) on VGG-11 DNN obfuscated by DNN-Alias as target model.

In this experiment, the adversarial samples are generated from the CIFAR10 dataset and the label output of VGG-11. We study this attack on the original (un-protected) DNN and then compare it with the obfuscated DNN. Also, we show the success rate of this attack on similar DNN families.

**1. Original Model:** To validate the adversarial attack implementation, we tested adversarial samples generated from the unprotected model extracted by SCAS. The results, represented by 1 in Fig. 16, show a success rate of \(98\%\) because the unprotected model closely resembles the original model.

**2. Obfuscated and Recovered Models:** Next, we generate the adversarial samples using the DNN obfuscated by DNN-Alias and test the target model. The results in 1 in Fig. 16 show that the adversarial attack on DNN-Alias is unsuccessful (success rate \(0.2\%\)). Furthermore, we launch NeuroUnlock [17] after the SCAS attack and we show 1 in Fig. 16) that the success rate increases to \(51\%\) on average. Although NeuroUnlock enhances the performance of the adversarial attack, the attack is still ineffective due to the errors in the de-obfuscation process.

**3. Public DNN Families:** We report the success rate of the adversarial attack on target DNN when the adversarial samples are generated using standard DNN families. The success rate for GoogleNet 1 is \(14\%\), Inception-V3 is \(48\%\), and ResNet-34 is \(88\%\). Since the attacker is unaware of the architecture of the target DNN to choose a similar DNN family, the results present that DNN-Alias successfully protects the DNN against SCAS-based adversarial attacks.

### _Overhead Analysis_

The results in Fig. 17 show that while DNN-Alias on average, increases memory access time for both read (\(13\%\)) and write (\(40\%\)) operations, the computation latency decreases (\(25\%\)). Thus, memory access time presents the primary bottleneck for further increasing the obfuscation level. This observation opens up opportunities for future research on the optimization of obfuscation techniques through the use of efficient memory protocols.

## VI Conclusion

In this paper, we present a novel obfuscation method called DNN-Alias to protect deep neural networks (DNNs) against

Fig. 14: The LER for the DNN-Alias and NeuroOfwscator obfuscated DNNs compared to the original model.

Fig. 13: The LER for the DNN-Alias obfuscated DNNs.

Fig. 15: Validation accuracy after training the models.
side-channel attacks. Our proposed method forces all the layers in a DNN to have similar execution traces, making it difficult for attackers to differentiate between the layers and extract the architecture. DNN-Alias employs a genetic algorithm to find the best combination of layer obfuscation operations to maximize the security level while maintaining a user-specified latency overhead budget.

The effectiveness of DNN-Alias is demonstrated through experiments on various randomly generated and publicly available DNNs. We show that DNN-Alias can successfully prevent state-of-the-art side-channel architecture stealing attacks and adversarial attacks while preserving the original functionality of the DNNs. Our results highlight the potential of DNN-Alias as a generic and hardware-independent defense mechanism for DNNs against side-channel attacks.
