Towards Invisible Backdoor Attacks in the Frequency Domain against Deep
  Neural Networks
Deep neural networks (DNNs) have made tremendous progress in the past ten
years and have been applied in various critical applications. However, recent
studies have shown that deep neural networks are vulnerable to backdoor
attacks. By injecting malicious data into the training set, an adversary can
plant the backdoor into the original model. The backdoor can remain hidden
indefinitely until activated by a sample with a specific trigger, which is
hugely concealed, bringing serious security risks to critical applications.
However, one main limitation of current backdoor attacks is that the trigger is
often visible to human perception. Therefore, it is crucial to study the
stealthiness of backdoor triggers. In this paper, we propose a novel
frequency-domain backdooring technique. In particular, our method aims to add a
backdoor trigger in the frequency domain of original images via Discrete
Fourier Transform, thus hidding the trigger. We evaluate our method on three
benchmark datasets: MNIST, CIFAR-10 and Imagenette. Our experiments show that
we can simultaneously fool human inspection and DNN models. We further apply
two image similarity evaluation metrics to illustrate that our method adds the
most subtle perturbation without compromising attack success rate and clean
sample accuracy.
Xinrui Liu, Yajie Wang, Yu-an Tan, Kefan Qiu, Yuanzhang Li
# Towards Invisible Backdoor Attacks in the Frequency Domain against Deep Neural Networks

###### Abstract

Deep neural networks (DNNs) have made tremendous progress in the past ten years and have been applied in various critical applications. However, recent studies have shown that deep neural networks are vulnerable to backdoor attacks. By injecting malicious data into the training set, an adversary can plant the backdoor into the original model. The backdoor can remain hidden indefinitely until activated by a sample with a specific trigger, which is hugely concealed, bringing serious security risks to critical applications. However, one main limitation of current backdoor attacks is that the trigger is often visible to human perception. Therefore, it is crucial to study the stealthiness of backdoor triggers. In this paper, we propose a novel frequency-domain backdooring technique. In particular, our method aims to add a backdoor trigger in the frequency domain of original images via Discrete Fourier Transform, thus hiding the trigger. We evaluate our method on three benchmark datasets: MNIST, CIFAR-10 and Imagenette. Our experiments show that we can simultaneously fool human inspection and DNN models. We further apply two image similarity evaluation metrics to illustrate that our method adds the most subtle perturbation without compromising attack success rate and clean sample accuracy.

Keywords:neural networks; backdoor attacks; frequency domain

## Introduction

With the advent of artificial intelligence, neural networks have become a widely used method of artificial intelligence. Currently, neural networks have been adopted in a wide range of areas, such as face recognition [1], voice recognition [2], games [3], and autonomous driving [4]. For example, PayPal users are using deep learning-based facial recognition systems to make payments. However, recent studies have shown that deep learning models are vulnerable to various attacks. Attacks against DNN [5] can be divided into three classes: adversarial example, poisoning attack, and backdoor attack. Adding some perturbation to the input data, an adversarial attack [6] can cause misclassification by the DNN without affecting the DNN. However, this attack generates perturbations specific to a single input. Poisoning attack [7] is a method that reduces the accuracy of the model by injecting malicious training data during the training phase. However, this method only reduces the accuracy of the model. Attackers cannot choose specific data they want to cause misclassification. Also, users will not deploy models with low accuracy under normal circumstances, which brings limitations in practice. To overcome these problems, backdoor attack [8] is proposed.

The backdoor attack enables attackers to plant a backdoor into the model and performs malicious at

Figure 1: Overview of our method. In the figure, DFT and IDFT represents Discrete Fourier Transform and Inverse Discrete Fourier Transform respectively. Note that we shift the zero-frequency component to the center of the spectrum.
tacks using a specific backdoor trigger in the inference phase. The backdoored deep neural network can correctly classify benign samples but will misclassify any input with a specific backdoor trigger as an attacker chosen target. The backdoor can remain hidden indefinitely until activated by a sample with a specific backdoor trigger, which is hugely concealed. Therefore, it can bring serious security risks to many critical applications.

Although backdoor attacks have been proven to cause neural network misclassifications successfully, one main limitation of current backdoor attacks is that backdoor triggers are usually visible to human perception. When the system administrator manually checks these datasets, the poisoned data will be found suspicious. [9] first discussed the importance of improving the stealthiness of backdoor triggers. They designed a method to blend the backdoor trigger with benign inputs instead of stamping the trigger as proposed in conventional backdoor attack [10][11]. After that, there was a series of researches dedicated to the invisibility in the backdoor attack. However, the backdoor inputs are still noticeable compared to benign samples, making existing backdoor triggers less feasible in practice. Therefore, improving the invisibility of backdoor triggers has become a research hotspot of neural network backdoor attacks. The challenge of creating an invisible backdoor is how to achieve smaller perturbation without affecting the attack success rate and clean sample accuracy. In 2019, [12] exploit the backdoor attack in a robust manner, namely hidden trigger backdoor. Here, the trigger is invisible to evade human inspections. However, we perform several experiments to prove that the perturbations they add are relatively large in contrast to our method. Besides, the adversary utilizes a neural network to optimize the original samples to generate poisoned samples, which raises the attack cost compared to our method.

It is well known that humans cannot perceive subtle variations in the color space within images. However, deep neural networks can detect slight perturbation due to their complexity and powerful feature extraction capabilities, making it possible to hide the trigger from manual review. Therefore, in this paper, we exploit this characteristic of DNNs to implement invisible backdoor attacks. Our method is motivated by the DFT-based image blind watermark. In this technique, a sender hides the covert information in the image frequency domain using an encoder. A receiver applies a decoder to extract the hidden message from the frequency domain to achieve secret messaging. According to our investigations, we are the first to propose the frequency-domain backdooring techniques. Figure 1 demonstrates an overview of our method. We add a backdoor trigger in the frequency domain of an original image to generate a poisoned sample which is invisible enough to evade human perception.

Our experimental results show that we can simultaneously achieve invisible backdoor attack without affecting attack success rate and clean sample accuracy. Also, we apply two image similarity evaluation metrics (\(l_{2}\) paradigm and LPIPS (Learned Perceptual Image Patch Similarity) [13]) to compare our method with the conventional method and a state-of-the-art hidden trigger attack [12]. We found that our method adds the smallest perturbation without compromising attack performance.

The contributions of this paper are as follows:

\(\bullet\) We propose the first class of frequency-domain backdooring techniques in which our method aims to add a backdoor trigger in the frequency domain of original images via Discrete Fourier Transform (DFT), thusidding the trigger.

\(\bullet\) We implement our DFT-based backdoor attack on MNIST, CIFAR-10, and a subset in Imagenet. Our experimental results show that our approach can simultaneously fool human inspection and DNN models.

\(\bullet\) We apply two image similarity evaluation metrics (\(l_{2}\) paradigm and LPIPS) to compare the invisibility of different methods. We find that our method adds the smallest perturbation without sacrificing attack success rate and clean sample accuracy.

The rest of the paper is organized as follows. Section 2 describes the related work. Section 3 explains the proposed scheme. Section 4 demonstrates the experimental setup and evaluates the results. Finally, we conclude the paper in Section 5.

## Related Work

Backdoor attack against DNNs

Backdoor attacks were first migrated to neural networks in 2017. [10] proposed BadNets. In this method, the attacker can attach a specific trigger to the stop sign image and mark it as the speed limit sign to generate a backdoor in the road sign recognition model. Although the model can correctly classify clean samples, it will misclassify the stop sign image with the trigger as the speed limit.

In 2018, [11] proposed a more advanced backdoor attack technique called Trojan attack. In the study of the Trojan attack, it was found that the backdoor attack method in the neural network was effective because the backdoor trigger would activate specific neurons in the network. Therefore, the Trojan attack generates a trigger in a way that maximizes the activations of specified neurons.

Based on classical backdoor attacks, many works focused on improving the invisibility of backdoor images.
[9] first discuss the importance of invisibility in backdoor attacks. They proposed that a backdoored image should be indistinguishable from its benign version to evade human inspection. To satisfy such a requirement, they generated poisoned images by blending the backdoor trigger with benign inputs rather than stamp the trigger as proposed in conventional backdoor attacks. After that, there was a series of researches dedicated to the invisibility in backdoor attacks. [14] proposed to utilize a backdoor trigger amplitude to perturb the clean images instead of replacing the corresponding pixels with the chosen pattern.

Interestingly, [12] exploit the backdoor attack in a robust manner, namely, hidden trigger backdoor. In this method, the trigger used in the poisoning phase is invisible to evade human inspections. However, we perform several experiments to prove that the perturbations they add are relatively large in contrast to our method, making it easily detected by programs. Besides, the attacker utilizes a neural network to optimize the original samples to add perturbations, which raises the attack cost to generate poisoned samples compared to our method.

In order to evaluate the invisibility of our method, we investigate a series of methods used to calculate image similarity, such as phash, \(l_{2}\) paradigm, \(l_{\infty}\) paradigm, and so on. Among them, LPIPS [13] is used to measure the similarity between two images in a manner that simulates human judgment. LPIPS is proposed based on perceptual loss. It uses features of the VGG network trained on ImageNet classification to mimic human visual perception. In this paper, we will use LPIPS as an invisibility evaluation metric.

### Blind Watermark

Blind watermark is an algorithm in steganography [15] which is the study of concealing information in plain sight, such that only the intended recipient would get to see it. Steganography encodes hidden messages onto conventional multimedia data, which may be an image, text, and video. One widely used algorithm in steganography is the Least Significant Bit (LSB) substitution. The idea behind LSB is that replacing bit 0 (i.e., the lowest bit) in a binary pixel value will not cause a visible change in the color space. Though this spatial-domain technique has the least complexity and high payload, it cannot withstand image compression and other typical image processing attacks, which bring poor robustness.

The frequency-domain blind watermark based on the Discrete Fourier Transform (DFT) [16] typically provides imperceptibility and is much more robust to image manipulations. The DFT-based blind watermark's main idea is to add a watermark image in the original image's frequency domain using DFT and transform the frequency-domain image back to spatial-domain using Inverse Discrete Fourier Transform (IDFT). Note that the frequency-domain image demonstrates the intensity of image transformation.

## Methodology

### Threat model

We assume a user who wants to use a training dataset \(D_{train}\) to train the parameters of a DNN. The user sends the internal structure of the DNN \(M\) to the trainer. Finally, the trainer will return to the user the trained model parameters \(\Theta^{{}^{\prime}}\).

However, the user cannot fully trust the trainer. The user needs to check the accuracy of the trained model on the validation dataset \(D_{valid}\). Only when the model's accuracy meets an expected accuracy rate \(r^{*}\) will the user accept the model.

**Attacker's Goals:** The attacker expects to return to

Figure 2: Two different DFT-based methods of generating poisoned samples for RGB images. The generation of Trigger B applies a RGB-to-Gray transformation, thus further improving the invisibility.
the user a maliciously trained backdoor model parameters \(\Theta^{{}^{\prime}}:=\Theta^{adv}\). The parameters of this model are different from those of the honestly trained model. A backdoored model needs to meet two goals:

Firstly, the classification accuracy of the backdoored model \(M_{\Theta^{adv}}\) cannot be reduced on the validation set \(D_{valid}\), in other words, that is, \(C(M_{\Theta^{adv}},D_{valid})\geq r^{*}\). Note that the attacker cannot directly access the user's validation dataset.

Secondly, for the input containing the backdoor trigger specified by the attacker, \(M_{\Theta^{adv}}\) outputs' predictions are different from the outputs of the honestly trained model.

#### 3.2.3 Generate poisoned images with DFT

In conventional backdoor trigger design approaches, the backdoor trigger is usually a distinct sign within an area, making backdoor data easily recognizable in the event of a human visual inspection. Our approach is inspired by the DFT-based image blind watermark [17] in image steganography [15]. Similarly, we add a trigger to an image's frequency domain so that the perturbation spreads throughout the image instead of being confined to a fixed region, thus making the trigger more invisible.

\[\begin{split} F(u,v)&=DFT(f(p,q))\\ &=\sum_{p=0}^{H-1}\sum_{q=0}^{W-1}f(p,q)e^{-i2\pi(\frac{vp}{H}+ \frac{vq}{W})}\end{split} \tag{1}\]

\[\begin{split} f(p,q)&=IDFT(F(u,v))\\ &=\frac{1}{HW}\sum_{u=0}^{H-1}\sum_{v=0}^{W-1}F(u,v)e^{i2\pi( \frac{vp}{H}+\frac{vq}{W})}\end{split} \tag{2}\]

We assume that we have a grayscale image that can be viewed as an \(H*W\) matrix (\(H\), \(W\) denote the height and width of the image, respectively). We can regard this image as a signal \(f(p,q)\) (denotes the pixel value of the spatial domain image at the coordinate point \((p,q)\)). In digital image processing, we usually utilize Discrete Fourier Transform (DFT) to convert an image from spatial domain to frequency domain. Besides, we apply \(F(u,v)\) to denote the pixel value of an image in frequency domain at the coordinate point \((u,v)\). The following Equation 1 represents Discrete Fourier Transform, and Equation 2 represents the Inverse Discrete Fourier Transform (IDFT), which transforms an image from frequency domain to spatial domain. Note that \(i\) denotes a unit of the complex number.

As shown in Algorithm 1: line 4 to line 8, we define a trigger \(F_{trigger}\) in frequency domain and the original image in spatial domain is represented as \(f_{original}\). We first convert the original image \(f_{original}\) to frequency domain using DFT (Equation 1), the result is represented as \(F_{original}\). Then, we add a trigger in the frequency-domain image \(F_{original}\) to generate a poisoned image of its frequency form. Here, we define an energy factor \(\alpha\) to indicate the strength of the trigger. The smaller the \(\alpha\), the lower the visibility of the trigger. Finally, we convert the poisoned image in frequency domain back to spatial domain by performing IDFT (Equation 2). \(f_{poisoned}\) is our generated spatial-domain backdoor image. Figure 1 demonstrates the visualization of our algorithm.

For RGB images, we design two approaches to add triggers in the frequency domain. One is to add the trigger directly in RGB-level frequency domain of the original image; the shape of the trigger is \(H*W*3\). The added perturbation is shown in Figure 2(a). In the second method, we first convert the RGB image to grayscale and then add a trigger (Note that the trigger shape here is \(N*M\)) in the grayscale frequency domain. Finally, we convert the gray image back to RGB-level, as shown in Figure 2(b).

#### 3.2.4 Backdoor injection

After generating DFT-based poisoned images, as shown in Algorithm 1: line 9, we replace the labels of the poison samples generated in Section 3.2 with the target label \(t\). After that, we can obtain a poisoned dataset \(D_{poisoned}\). We apply the poisoned dataset \(D_{poisoned}\) with the clean dataset \(D_{clean}\) to retrain the model parameters \(\Theta^{adv}:=\Theta^{{}^{\prime}}\).

In the inference phase, we apply the same frequency-domain trigger and \(\alpha\) value used in the training phase to generate poisoned validation samples. After that, we record the Clean Sample Accuracy (CSA) as well as the Attack Success Rate (ASR) to evaluate our attack. We will show our experiment results in the next section.

## 4 Experiments and Analysis

#### 4.0.1 Experiment setup

In this section, we implement the DFT-based backdoor attack introduced in Section 3.

Datasets and models.For the DFT-based backdoor attack, we mount our attack on MNIST [18], CIFAR-10 [19], and Imagenette which is a subset in ImageNet [20]. All datasets are widely used in deep learning. Our experiments were run on a machine with two 2080Ti, and our networks are implemented by Pytorch 1.5 [21]. For MNIST digit recognition task, in order to obtain high classification accuracy, we use AlexNet [22] as our baseline model. For CIFAR-10 and
Imagenette, we use pre-trained ResNet-18 [23] as the original model. Note that we use Adam [24] on Alexnet with a learning rate of \(1e-3\) and apply SGD [25] optimizer on ResNet-18 with a learning rate of \(1e-2\).

Evaluation metric.The success of a backdoor attack can be generally evaluated by Clean Sample Accuracy(CSA) and Attack Success Rate(ASR), which can be defined as follows:

**Clean Sample Accuracy (CSA):** For normal users, the CSA measures the proportion of clean test samples containing no trigger that is correctly predicted to their ground-truth classes.

**Attack Success Rate (ASR):** For an attacker, we represent the output of the backdoored model \(M_{\Theta^{adv}}\) on poisoned input data \(x^{poisoned}\) as \(y^{\prime}=M_{\Theta^{adv}}(x^{poisoned})\) and the attacker's expected target as \(t\). This index measures the ratio of \(y\) which equals the attacker target \(t\). This measurement also shows whether the neural network can identify the trigger pattern added to the input images.

#### DFT-based backdoor attack

In order to construct the poisoning training dataset with our DFT-based algorithm, we inject the frequency-domain trigger into 10% training data. For the images in which we plant the trigger, we replace their labels with our target label. In MNIST, CIFAR-10, and Imagenette, we select digit 5, "deer", and "building" as our targets respectively. We apply an energy factor \(\alpha\) to control the invisibility of the poisoned images. To make the neural network learn the features of our frequency-domain trigger, we retrain the baseline models on the poisoning dataset with a small learning rate. When validating the backdoored model, we hide our trigger on the original validation dataset using the same \(\alpha\) value, and then we compute their Clean Sample Accuracy (CSA) and Attack success Rate (ASR) (see **Section 4.1**).

DFT-based method for gray images.First, to demonstrate the feasibility of our attack, we conduct experiments on MNIST. Figure 3 shows the poisoned samples generated on MNIST using different \(\alpha\) values, and the first image shows the highlighted trigger pattern generated by our method for grayscale images. Table 1 shows the performance of our attack on MNIST using different \(\alpha\) values. During the process of our experiments, we find that the smaller the \(\alpha\) value, the slower the model converges and the more epochs are needed for training, which indicates that it is more difficult for our model to capture such slight perturbations. However, both ASR and CSA end up close to 100%. Additionally, the \(l_{2}\) value of the perturbation at \(\alpha=0.1\) reaches only 0.0122 without affecting the performance, which means our model can detect the subtle change in image's frequency domain, thus making the trigger invisible.

DFT-based method for RGB images.To evaluate the performance of the attack on the trigger strength of poisoned samples on CIFAR-10 and Imagenette, we carried out extensive experiments which are summarized in Figure 5. According to our two methods of crafting poisoned samples for RGB images proposed in Figure 2, we set different \(\alpha\) values on CIFAR-10 and Imagenette to perform several backdoor attacks and test the attack success rate(ASR) as well as clean

\begin{table}
\begin{tabular}{l c c c} \hline \hline  & \(\alpha=0.1\) & \(\alpha=0.5\) & \(\alpha=1\) \\ \hline Epoch & 48 & 10 & 3 \\ CSA & 98.53\% & 98.31\% & 98.89\% \\ ASR & 98.48\% & 99.99\% & 99.99\% \\ \(l_{2}\) & 0.0122 & 0.0610 & 0.1219 \\ \hline \hline \end{tabular}
\end{table}
Table 1: DFT-based backdoor attack performance and \(l_{2}\) values for different \(\alpha\) values on MNIST.

Figure 3: The figure shows the spatial trigger, original image and the poisoned samples generated by DFT-based method with \(\alpha=0.1\), \(\alpha=0.5\) and \(\alpha=1\) respectively.
sample accuracy(CSA). Figure 4 shows the generated poisoned samples using different alpha values of two triggers. From four subfigures in figure 5, we can see that the ASR generally increases by boosting \(\alpha\). Besides, in figure 5(a)(b), even when \(\alpha=0.3\), the poisoned samples can be misclassified as our target with accuracy larger than \(90.0\%\). The effectiveness of conventional backdoor attack can be further enhanced by considering our method with \(\alpha=0.3\) on CIFAR-10, which still guarantee the attack concealment without compromising the ASR and CSA. As for Imagenette, the best tradeoff point is \(\alpha=0.5\) for Trigger A and \(\alpha=1\) for Trigger B.

Besides, we perform experiments to compare the two DFT-based methods for RGB images on Imagenette, which are summarized in Table 2. In the table, the "Best \(\alpha\)" indice indicates the lowest \(\alpha\) values for Trigger A and Trigger B, respectively, while ensuring the ASR and CSA. Additionally, we apply \(l_{2}\) paradigm and LPIPS to evaluate the invisibly of the two methods. From the table, we find that in contrast to Trigger A, Trigger B has better invisibility without sacrificing ASR and CSA. Figure 7 illustrates the accuracy of the backdoored model on the clean images (CSA) and the validation poisoning dataset (ASR). From the figure, it is clear that we can stealthily achieve our DFT-based backdoor attack on MNIST (\(\alpha=0.1\)), CIFAR-10 (\(\alpha=0.3\)) and Imagenette (\(\alpha=1\)) while hiding the trigger from human perception, indicating that our backdoored model can accurately identify the subtle changes in the image's frequency domain and simultaneously achieve the misclassification of the network.

### Comparison with classical attack

We also conducted several experiments to compare our two methods with classical backdoor attack. Figure 6 shows different backdoored samples and their corresponding triggers. To prove the stealthiness of our method, we compute \(l_{2}\) values and LPIPS indices of the four types of triggers used in classical backdoor [11] and two DFT-based method for RGB images proposed in figure 2. For our two methods, we select \(\alpha\) values used in table 2.

\(l_{2}\) value is used to calculate the euclidean distance between the backdoored image and the original image, so a lower value indicates the images are more similar. Recall that the LPIPS score measures the perceptual distance between the reference image and the blurred image. The range of LPIPS score is \([0,1)\). If two images are identical, the value is 0. A lower LPIPS value means two images are more similar; a higher score means the images are more different. A comparison of the \(l_{2}\) paradigm value and LPIPS score for each attack is illustrated in Table 3. Our method achieves lower \(l_{2}\) value and LPIPS (near 0). This demonstrates that it is more difficult for humans to distinguish our poisoned images from original images.

### Availability of data and materials

The dataset analysed during the current study was taken from [https://github.com/VinAIResearch](https://github.com/VinAIResearch).

### Funding

Acknowledgements

## Conclusion

In this paper, we propose a novel method to add the backdoor trigger in the frequency domain of original images to generate poisoned samples. The poisoned data looks similar to the original images and does not reveal the trigger. Therefore, it is invisible enough to evade the event of a human visual inspection. Experiments on three different datasets demonstrate that our

\begin{table}
\begin{tabular}{c c c c c c} \hline \hline Trigger & Best \(\alpha\) & ASR & CSA & \(l_{2}\) & LPIPS \\ \hline A & 0.5 & 90.14\% & 95.26\% & 1.057 & 1.9e-3 \\ B & 1 & 89.11\% & 95.06\% & 0.914 & 7.8e-4 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Comparison between Trigger A and Trigger B proposed in figure 2.

Figure 4: The figure shows trigger, original image and the poisoned samples generated by DFT-based method using different \(\alpha\) values on CIFAR-10 (row \(1,2\)) and Imagenette (row \(3,4\)). Row \(1,3\) and row \(2,4\) respectively demonstrates two different triggers proposed in figure 2.
method implements invisible backdoor attacks without compromising the ASR and CSA. Additionally, we use two image similarity evaluation metrics to compare our method with a conventional backdoor attack and a state-of-the-art hidden trigger backdoor attack. We find that our approach adds the smallest perturbation. We believe such invisible backdoor attacks reveal the vulnerabilities of deep neural networks that need to be deployed in critical real-world applications.

## Author details

School of Cyberspace Science and Technology, Beijing Institute of Technology, Beijing, China.
