In recent years, deep neural networks have demonstrated outstanding performancein many machine learning tasks. However, researchers have discovered that thesestate-of-the-art models are vulnerable to adversarial examples:  legitimate examples added by small perturbations which are unnoticeable to human eyes. Adversarial training, which augments the training data with adversarial examples duringthe training process,  is a well known defense to improve the robustness of themodel against adversarial attacks.   However, this robustness is only effective tothe same attack method used for adversarial training.   Madry et al. (2017) suggest that effectiveness of iterative multi-step adversarial attacks and particularlythat projected gradient descent (PGD) may be considered the universal first order adversary and applying the adversarial training with PGD implies resistanceagainst many other first order attacks.    However,  the computational cost of theadversarial training with PGD and other multi-step adversarial examples is muchhigher than that of the adversarial training with other simpler attack techniques. In this paper, we show how strong adversarial examples can be generated only ata cost similar to that of two runs of the fast gradient sign method (FGSM), allowing defense against adversarial attacks with a robustness level comparable to thatof the adversarial training with multi-step adversarial examples.   We empiricallydemonstrate the effectiveness of the proposed two-step defense approach againstdifferent attack methods and its improvements over existing defense strategies. Despite the fact that deep neural networks demonstrate outstanding performance for many machine learning tasks, researchers have found that they are susceptible to attacks by adversarial examples BID18 ; BID2 ). Adversarial examples which are generated by adding crafted perturbations to legitimate input samples are indistinguishable to human eyes. For classification tasks, these perturbations may cause the legitimate samples to be misclassified by the model at the inference time. While there exists no widely agreed conclusion, several studies attempted to explain the underlying causes of the susceptibility of deep neural networks toward adversarial examples. The vulnerability is ascribed to the linearity of the model BID2 ), low flexibility BID1 ), or the flatness/curvedness of the decision boundaries BID10 ), but a more general cause is still under research. The recent literature considered two types of threat models: black-box and white-box attacks. In black-box attacks, the attacker is assumed to have no access to the architecture and parameters of the model, whereas in white-box attacks, the attacker has complete access to such information. Several white-box attack methods were proposed BID2 , BID12 , BID17 , BID0 , BID9 ). In response, several defenses have been proposed to mitigate the effect of adversarial attacks. These defenses were developed along three main directions: (1) expanding the training data to make the classifier more robustly learn the underlying function, e.g., by adversarial training which augments the training data set with adversarial examples generated by certain attack methods BID18 , BID2 , BID5 ); (2) modifying the training procedure to reduce the gradients of the model w.r.t. the input such that the classifier becomes more robust to input perturbations, e.g., via input gradient regularization BID15 , or defensive distillation BID14 ; and (3) using external models as network add-ons when classifying unseen examples (feature squeezing BID19 , MagNet BID8 , and Defense-GAN) BID16 ).Adversarial training, a simple but effective method to improve the robustness of a deep neural network against white-box adversarial attacks, uses the same white-box attack mechanism to generate adversarial examples for augmenting the training data set. However, if the attacker applies a different attack strategy, adversarial training does not work well due to gradient masking BID13 . BID7 have suggested the effectiveness of iterative multi-step adversarial attacks. In particular, it was suggested that projected gradient descent (PGD) PGD may be considered the strongest first-order attack so that the adversarial training with PGD can boost the resistance against many other first-order attacks. However, in the literature a large number (e.g. 40) of steps of back propagation are typically used in the iterative attack method of PGD or its closely related variant iterative fast gradient (IFGSM) BID5 to find strong adversarial examples to be used in each adversarial training step, incurring a prohibitively high computational complexity particularly for large DNNs or training datasets. In this paper, we propose an efficient two-step adversarial defense technique, called e2SAD, to facilitate defense against multiple types of whitebox and blackbox attacks with a quality on a par with the expensive adversarial training using the well-known multi-step attack the iterative fast gradient method (IFGSM) BID5 . The first step of e2SAD is similar to the basic adversarial training, where an adversarial example is generated by applying a simple one-step attack method such as the fast gradient sign method (FGSM). Then in the second step, e2SAD attemps to generate a second adversarial example at which the vulnerability of the current model is maximally revealed such that the resulting defense is at the same quality level of the much more expensive IFGSMbased adversarial training. Finally, the two adversarial examples are taken into consideration in the proposed loss function according to which a more robust model is trained, resulting strong defense to both one-step and multi-step iterative attacks with a training time much less less than that of the adversarial training using IFGSM. The main contributions of this paper are as follows:• We propose a computationally efficient method to generate two adversarial examples per input example while effectively revealing the vulnerability of the learned classifier in the neighborhood of each clean data point;• We show that by considering the generated adversarial examples as part of a well-designed final loss function, the resulting model is robust to both one-step and iterative white box attacks;• We further demonstrate that by adopting other techniques in our two-step approach like the use of soft labels and hyper parameter tuning, robust defense against black box attacks can be achieved. We provide a brief overview of related existing attacks and defense methods, part of which will be also used to compare with the proposed e2SAD approach. Attack Models and Algorithms. The goal of all attack models is to find a perturbation δ to be added to a clean input x ∈ R d , resulting in an adversarial example x adv = x + δ which may potentially lead to misclassification of the classifier. Typically, the noise level of the perturbation is constrained by the ∞ ball denoted by ε to make sure that the perturbation is sufficiently small. Based on the amount of information the attacker knows, there are two threat levels as follows:1. White box: the attacker has full information about the model including its architecture and parameters such that it is possible craft adversarial examples using techniques such as gradient based attacks to specifically target the model;2. Black box: the attacker has no knowledge about the architecture and parameters of the model. Neither is the attacker able to query the model. Adversarial examples can be generated using a substitute model which is a white-box to the attacker. The Fast Gradient Sign Method (FGSM). Given a clean input x and its corresponding true label y, FGSM perturbs x by BID2 : DISPLAYFORM0 where J(θ, x, y) is the loss function and ε ∈ [0, 1] is a constant value used to constrain the noise level of the perturbation. Iterative Fast Gradient Sign Method (IFGSM) and PGD IFGSM attack generates adversarial examples by iteratively applying FGSM attack multiple, say N , times to a clean input x with a small constant a BID5 ) DISPLAYFORM1 In our implementation, we set a = ε N . Typically, each component of the input vector, e.g. a pixel, is normalized to be within [0, 1] . The function clamp x,ε is an elementwise clipping function which clips each element x i of input x into the range of [max(0, DISPLAYFORM2 Projected gradient descent (PGD) is a closely related variant of IFGSM. Typically, PGD first randomly picks a point within a confined small ball around each clean input and then applies the multistep IFGSM to generate adversarial examples for that clean input. Adversarial Training. This is a popular defense approach which augments the training dataset with adversarial examples BID2 , BID5 ). In our implementation, we adopt the adversarial training equation proposed in BID2 as the loss function DISPLAYFORM0 where α is a constant specifying the relative importance of the adversarial examples. In our latter comparison, we choose two methods, FGSM and IFGSM, for generating the adversarial examples. Minmax. There exist defense methods BID4 ; BID7 ) which view the process of training a robust model as solving a minmax optimization problem DISPLAYFORM1 where D is the underlying training data distribution, J(θ, x, y) is the loss function, and θ is the parameters of the model. In BID4 ) and BID7 ), the maximization with respect to δ is approximated by a specific attack method, for example, by PGD Madry et al. (2017) .Hamm FORMULA0 proposes a new approach which can instead of targeting the saddle points like previous methods, find the true optimal solution of the minmax optimization problem. BID3 chooses FGSM to approximate the inner maximization step and for the outer minimization step, instead of plugging the adversarial version of the clean data directly and solving the optimization problem, changes the minimization objective function to DISPLAYFORM2 where ε is a constant restricting the level of input perturbation, and N is the number of the training examples in each minibatch. Adversarial training, which augments the training dataset with adversarial examples during the training process, has been shown to increase the robustness of the model against white box attacks when the attack method used to generate the augmented training set is the same as the method used by the attacker. However, if the attacker uses a different attack strategy to apply the white box attack, adversarial training does not perform well. For example, adversarial training using one-step FGSM can not improve the robustness of the model against multi-step attacks such as IFGSM and PGD. However, compared to adversarial training using IFGSM or PGD, adversarial training using one-step FGSM takes much less time for the training process since it takes only one step of back propagation to generate adversarial example during each training iteration. BID7 suggest that PGD, one particular type of multi-step iterative adversarial attack, is the strongest universal first-order adversary. It is also suggested that the model trained by the adversarial training with PGD is robust against both PGD and one-step FGSM, however at the expenses of multiple steps of back propagation per a clean training data point. Our ojective is to develop a defense method with a cost similar to that of FGSM adversarial training while being robust to both FGSM and multiple-step attacks such as IFGSM. A shown in FIG0 , the proposed efficient two-step adversarial defense (e2SAD) approach takes only two steps of back propagation to find adversarial examples. First, for a given input we define the categorical distribution of the model as the vector of probabilities the model outputs, where each component of the vector representing the probability for the input to be in the corresponding class. At the first step of e2SAD, a one-step attack method such as FGSM is applied to find the first adversarial example per each clean input. At the second step, within the neighborhood of the first adversarial example, the input point whose categorical distribution is most different from that of the first adversarial point is selected as the second adversarial example. For each clean input, these two generated adversarial examples are considered in the final loss function for training. The loss function consists of three terms: the loss of the original clean inputs, the loss of the adversarial examples generated at the first step, and the dissimilarity in categorical distribution of all pairs of the corresponding first and second adversarial examples. It is worth noting that the two-step e2SAD approach is structured in a particular way such that it may provide strong defense against both onestep and multiple-step attacks, as detailed below. The main objective of the first step of e2SAD is to find a highly vulnerable neighborhood immediately around each clean training data point such that the trained model can be made robust to one-step gradient-based attacks. In so doing, we simply apply a one-step attack method such as FGSM to maximize the loss around around each clean input x i to generate the first adversarial example x adv i DISPLAYFORM0 where ε 1 is a constant chosen step size. We include the loss of this adversarial example in the final loss function (8), discussed in detail in the next subsection. Essentially, by doing so, the first term of (8) guides the training process to reduce the losses of both x i and x adv i , acting as a mechanism for defending one-step adversarial attacks. As discussed earlier, compared to one-step adversarial attacks iterative multi-step attacks can be much stronger as they search the neighborhood of a clean data point more exhaustively, which in turns makes the adversarial training using iterative adversarial attacks a stronger defense. At the second step of e2SAD, our goal is to efficiently defend against multi-step attacks by using only one extra step of computation. As such, the key challenge here is to find a second adversarial examplẽ x adv i which is close to x adv i and can effectively reveal the vulnerability of the model in a way similar to expensive multi-step attacks. In a multi-step attack method such as IFGSM or PGD, each adversarial example in the iterative process is typically found by perturbing the preceding adversarial example to maximize its loss, where the loss, for example, may be described using the cross entropy based on either the hard or soft label. Despite this common practice, we argue that a more appropriate approach is to instead constrain the training process such that a level of similarity (or uniformity) in the prediction of the trained model is maintained in the neighborhood of each clean input x i . It is important to note that maintaining similarity of prediction and minimizing the loss may be correlated but are not necessarily identical objectives; the latter attempts to ensure that predictions made in some neighborhood of the input individually have low loss without specifically constraining these predictions to be similar to each other. Nevertheless, we believe that the objective of maintaining similarity of prediction is more relevant as far as adversarial defense is concerned as it may lead to a well-regularized decision boundary around each x i .With the above understanding, at the second step of e2SAD, we attempt to find the second adversarial examplex adv i whose categorical distribution is maximally different from that of the first adversarial example x adv i in the neighborhood of x adv i . The dissimilarity in categorical distribution between these two points is measured by cross entropy (CE). To locatex adv i , FGSM is used as a one-step optimization method to maximize the CE-based dissimilarity measurẽ DISPLAYFORM0 where ε 2 is the step size, and the gradient of the CE-based dissimilarity is evaluated at x adv i . The reason for using categorical distribution as the measure of dissimilarity to find the second adversarial point is as follows. First, note that the value of loss for a model prediction does not fully indicate whether the prediction is a misclassification or not. To see this, consider a simple classification task with three classes. Assume that the true class labels for two different inputs are both the first class, and the corresponding categorical distributions are [0.45, 0.55, 0] and [0.4, 0.3, 0.3], respectively. Let us further assume that one-hot encoding is conventionally used for the labels. In this case, the model misclassifies the first input while correctly classifies the second. However, this happens even when the loss of the first input is lower than that of the second input. Figure 2 shows how the choice of the optimization objective may influence the generation of the second adversarial example for an illustrative three-class classification problem. The probabilities of three classes predicted by a trained model for a set of inputs are illustrated using the green, red, and purple curves, respectively. Accordingly, the one-hot encoding loss as a function of the input is shown by the blue curve. The cross entropy of categorical distribution between each input and Figure 2 : Using the loss and cross-entropy dissimilarity at the second step of e2SAD for a three-class classification problem. Both the clean input and first adversarial example belong to class 1. Green cross: the 1st adversarial point; Red cross: the 2nd adversarial point found by applying FGSM on the loss value; Yellow cross: the 2nd adversarial point found by applying FGSM on the cross entropy dissimilarity with the 1st adversarial point. Only the yellow cross corresponds to a misclassification. the first adversarial example (green cross) is shown by the orange curve. Note that both the clean input and first adversarial example belong to class 1 in this setup. Starting from the green cross, maximizing the loss using FGSM produces the red cross as the second adversarial example. In comparison, using the CE dissimilarity measure as the objective function leads to the yellow cross. While having the highest loss, the red cross is correctly classified by the model. On the other hand, misclassification happens at the yellow cross which is found based on the CE dissimilarity measure, suggesting its effectiveness in finding stronger adversarial points. Based the two adversarial examples generated at the two steps of e2SAD, we design the loss function used for training the final model as follows. For a mini batch X of m clean examples {x 1 , · · · , x m } and the corresponding mini batch of the first set of adversarial examples X adv = {x adv 1 , · · · , x adv m } generated at the first step of e2SAD, the total loss function is given by DISPLAYFORM0 where θ is the parameters of the model, eachx adv i is the second adversarial example which has the maximally different categorical distribution from the corresponding first adversarial example x adv i , f (x, θ) indicates the categorical distribution output function of the model for input x, and D is the cross-entropy dissimilarity measure. We adopt label smoothing BID18 for the training process. Here, instead of using hard labels (one-hot labels) for each cross-entropy loss, we employ the so-called soft labels which assign the correct class a target probability of 1 − δ and divide the remaining δ probability mass uniformly among the incorrect classes. We have found that the use of label smoothing in e2SAD leads to better performance. The overall training algorithm of the proposed e2SAD approach is summarized in Algorithm 1. The hyperparameters α and λ specify the weights for the losses of the clean and first set of adversarial inputs and those for the dissimilarity between each pair of the first and second adversarial inputs, respectively. While the first two terms in the final loss function target the defense against one-step adversarial attacks, the last term mainly plays the role of defending multi-step attacks. In practice, α and λ shall be properly chosen to balance between these two different defense needs. Input: training dataset (X, Y ); Initial model parameter θ; batch size: m; hyperparameters α, λ, ε 1 , ε 2 Output: Trained model parameter θ 1: for each minibatch t do 2:for each (x i , y i ) in the current batch do 3: DISPLAYFORM0 end for 6: DISPLAYFORM1 7: DISPLAYFORM2 Update θ using backpropogation based on L total 10: end for 11: Return θ We visually show the two-step e2SAD adversarial example generation process and the loss surfaces of four different models for a minibatch of 128 clean images from the MNIST handwritten digits dataset BID6 in FIG2 and FIG3 of the Appendix, respectively, to demonstrate the effectiveness of e2SAD. We compare the proposed e2SAD method with two widely adopted techniques in the literature: adversarial training using single-step FGSM BID2 and the adversarial training using multi-step IFGSM. We also report our experience on the minimax adversarial defense method proposed in BID3 . We adopt the widely used the MNIST handwritten digits dataset (LeCun et al., 1998) and the Street View House Numbers (SVHN) Dataset BID11 as benchmarks. MNIST consists 60,000 training images and 10,000 testing images, where each pixel value is normalized to be within [0, 1]. The adversarial attacks considered are:• White-box attacks with FGSM under different noise levels: ε = 0.3, 0.4.• White-box attacks with IFGSM under the fixed noise level of ε = 0.3 with different numbers of steps: k = 10, 30.• Black-box attacks from three substitute models: the naturally trained model (i.e. the one trained using only the clean inputs without any additional defense strategy), one trained with FGSM adversaries under the noise level of ε = 0.3, and one trained with IFGSM adversaries under the total noise level of ε = 0.3 and step size of 0.01 (k = 30). With respect to these substitute models, IFGSM with ε = 0.3 and k = 30 is used to generate adversarial examples, which are then employed to attack each of the targeted models. All CNNs we use consist of two convolutional layers with 32 and 64 filters respectively, each of which is followed by a 2 × 2 max-pooling layer and ReLU activation function, and a fully connected layer of 1, 024 neurons. The configuration of the CNNs is summarized in TAB5 in the Appendix. For our proposed e2SAD method, we set the hyperparameters in the training Algorithm 1 as: α = 0.6, λ = 0.1, ε 1 = 0.3, and ε 2 = 0.1. To increase the searching ability of the second step of e2SAD, we do not clamp the second adversarial point to be within a norm ball around the clean data point. All models are trained on MNIST for 30,000 iterations with the batch size of 256.Comparison with adversarial training The performances of different models under various white-box attacks are shown in Table 1 . It can be seen that each model reaches the accuracy of over 99% on the clean dataset. The baseline model trained naturally shows no defense ability towards both FGSM and IFGSM adversaries while other three models demonstrate different levels of defense. The model obtained via FGSM adversarial training maintains a very high accuracy under FGSM attacks with different noise levels. However, FGSM adversarial training can only defend FGSM attacks while shows no defense ability against IFGSM attacks of any step number. IFGSM adversarial training performs well under IFGSM adversaries and also shows robustness against FGSM attacks. However, the defense ability drops fast when the noise level ε increases in the case of FGSM attacks. Specifically, the accuracy can drop by almost 14% under the FGSM attacks when the noise level increases to ε = 0.4. Note that it makes 30 steps to generate IFGSM adversarial examples in each training iteration, leading to the high cost of the considered IFGSM adversarial training. Among all models considered, the proposed e2SAD method produces the highest accuracy for both the clean data and FGSM attacks at different noise levels. Under IFGSM attacks e2SAD significantly outperforms the FGSM adversarial training, demonstrating the effectiveness of the proposed twostep approach's generalization capability with respect to defense against strong multi-step attacks. Compared with the adversarial training using IFGSM, e2SAD offers stronger defense against FGSM attacks while maintaining a good robustness against IFGSM attacks. Note that these are achieved using only two steps of gradient calculation in each training iteration, presenting a significant reduction of computational cost compared with the IFGSM adversarial training, which performs 30 steps of gradient computation. Label smoothing is adopted in e2SAD and it is shown to be effective in helping the trained model generalize well. In our experiments, we set the probability for the correct label to 0.75 and the one for all other incorrect labels to 0.25. Table 1 shows that label smoothing also improves the performance of the traditional adversarial training under some circumstances, but not significantly. Comparison with the minimax adversarial defense We also implemented the minimax adversarial defense method proposed in BID3 ) with a minor modification that the model is trained using a mixture of clean and adversarial examples to achieve better performance. Our results show that the trained model is very robust against FGSM attacks, however, shows no defense against IFGSM attacks. In TAB2 , we consider how adversarial examples generated by applying IFGSM to a substitute model may attack a different model. The rows of the table are the considered substitute models: "Natural model" is again the baseline model without any additional defense strategy; "FGSM ε = 0.3" is the model obtained via FGSM adversarial training with the setting ε = 0.3; "IFGSM ε = 0.3, k = 30" is the model obtained via IFGSM adversarial training with the setting ε = 0.3, k = 30. The substitution models are trained using hard labels ("H") and label smoothing ("S"), then attacked by IFGSM with the setting (ε = 0.3, k = 30) for generating adversarial examples. The adversarial examples generated from the substitute models are used to attack the four models shown in the columns of the table: "Natural" is the baseline model; "FGSM Adv. Train" and "IFGSM Adv. Train" are models trained by the FGSM and IFGSM adversarial training using the settings specified in the table, respectively; "e2SAD" is the proposed model. The models under attack are trained using both hard and label smoothing except for e2SAD which is based on label smoothing only. Note that in TAB2 , white-box attacks are resulted when the substitute model and the one under attack are identical, and all other combinations correspond to black-box attacks. TAB2 demonstrates that the proposed e2SAD approach delivers a well-balanced defense against black-box IFGSM attacks from all three substitute models with an accuracy of nearly 90% or higher. There are several cases under which the natural training (baseline) and FGSM adversarial training have a poor performance. In all cases, e2SAD either noticeably outperforms both the natural training and FGSM adversarial training or produce a fairly close performance. Compared with the models trained with the 30-steps IFGSM adversarial training, e2SAD is still very competitive particularly given the fact that only two-steps of gradient computation are performed at each training iteration. The Street View House Numbers (SVHN) dataset ( BID11 ) consists of a training set of 73,257 digits and a testing set of 26,032 digits obtained from house numbers in Google Street View images, representing a significantly harder real-world dataset compared to MNIST. We process the SVHN dataset by removing the mean and normalizing the pixel values with the standard deviation of all pixels in each image so that the normalized pixel values are within [-1, 1] .We train three different models with the CNN configuration summarized in TAB6 in the Appendix and compare their performances under the scenario of white box attacks. All models are trained for 20 epochs with the following setup• FGSM-based adversarial training: {Batch size = 256, optimizer=AdamOptimizer with learning rate 0.001, α = 0.6, ε = 24/255} • IFGSM-based adversarial training: {Batch size = 256, optimizer=AdamOptimizer with learning rate 0.001, α = 0.6, ε = 24/255, attack steps=10} • e2SAD: {Batch size = 256, optimizer=AdamOptimizer with learning rate 0.001, α = 0.6, λ = 0.3, ε 1 = 24/255, ε 2 = 8/255, label smoothing with correct class probability of 0.75}The performances of the various models on this much harder SVHN dataset are summarized in TAB3 . It turns out that e2SAD outperforms all other models in this case. More specifically, the baseline (natural) model shows no defense to any attack. e2SAD attains a significantly stronger robustness against the iterative IFGSM white-box attacks compared with the FGSM adversarial training, which shows no defense to such attacks. Furthermore, compared with the expensive IFGSM adversarial training, e2SAD offers a much stronger defense against the one-step FGSM attacks. This fact may be attributed to the particular two-step structure of e2SAD, which is geared towards defending both one-step and multi-step adversarial attacks. We have aimed to improve the robustness of deep neural networks by presenting an efficient twostep adversarial defense technique e2SAD, particularly w.r.t to strong iterative multi-step attacks. This objective is achieved by finding a combination of two adversarial points to best reveal the vulnerability of the model around each clean input. In particular, we have demonstrated that using a dissimilarity measure between the first and second adversarial examples we are able to appropriately locate the second adversary in a way such that including both types of adversaries in the final training loss function leads to improved robustness against multi-step adversarial attacks. We have demonstrated the effectiveness of e2SAD in terms of defense against while-box one-step FGSM and multi-step IFGSM attacks and black-box IFGSM attacks under various settings.e2SAD provides a general mechanism for defending both one-step and multiple attacks and for balancing between these two defense needs, the latter of which can be achieved by properly tuning the corresponding weight hyperparameters in the training loss function. In the future work, we will explore hyperparameter tuning and other new techniques to provide a more balanced or further improved defense quality for a wider range of white and black box attacks. To demonstrate the two-step adversarial generation process of e2SAD, we consider a minibatch of 128 clean images from the MNIST handwritten digits dataset BID6 . We apply e2SAD to find the first and second adversarial examples for each clean image x i in the batch. To help visualize the loss surface of the model around this minibatch, which may be explored by IFGSM attacks in a two-dimensional input space, we identify a search direction g 1 = sign(x adv,IF i − x i ), where x adv,IF i is the adversary for x i found by IFGSM. We define a second search direction g 2 to be orthogonal to g 1 . Then around each x i , we generate a set of perturbed images along g 1 and g 2 : 0, 0.4] . t 1 and t 2 are chosen to be the two lateral axes in FIG2 . Here the loss is defined as the cross entropy loss based on hard target labels. The mesh loss surface shows the loss of the model summed over the perturbed images for the entire minibatch as a function of t 1 and t 2 . The blue dot at (0, 0) location is the loss of the minibatch of clean images. The red line starting from this blue point illustrates the two-step e2SAD adversarial searching direction. The second and third blue points on the red line show the losses summed over the first and second sets of adversarial examples, respectively, generated by e2SAD for this minibatch of clean images. The locations of these two points are projected on the t 1 and t 2 coordinates for visualization. In this case, at the second step e2SAD is able to identify an effective set adversarial examples with a cost further increased from the first set, suggesting its effectiveness in defending both one-step and multi-step adversarial attacks. DISPLAYFORM0 We visualize the loss surfaces of different models to shed light on the potential defense capabilities of these models with respect to both one-step FGSM attacks and multi-step IFGSM attacks in FIG3 and FIG3 , respectively. Here, the baseline model again is only trained with the clean data and with no additional defense strategy; "FGSM Adv. Train" is the model is trained by adversarial training with adversaries generated from FGSM (ε = 0.3); "IFGSM Adv. Train" is the model trained by adversarial training with adversaries generated from IFGSM (ε = 0.3, k = 30); And e2SAD is the proposed approach with the setting (θ = 0.2, λ = 0.1, ε = 0.3, ε = 0.1). All models are trained using a total number of 30,000 mini-batches of 256 images each over the MNIST dataset. FIG3 and FIG3 illustrate the loss surface of each model in the input space, which may be viewed by FGSM and IFGSM attacks, respectively, when they generate adversarial examples. To make visualizations possible in a reduced 2-dimensional input space, we take the approach adopted in FIG2 . For example, in the case of FIG3 , we identify a search direction g 1 = sign(x adv,F GSM i − x i ), where x adv,F GSM i is the adversary for each clean image x i found by the FGSM attack. We define a second search direction g 2 to be orthogonal to g 1 . Then around each x i , we generate a set of perturbed images along g 1 and g 2 : X p = {x i + t 1 · g 1 + t 2 · g 2 }, t 1 , t 2 ∈ [0, 0.4]. t 1 and t 2 are again chosen to be the two lateral axes in FIG3 as in FIG2 . The mesh loss surface of a model shows the loss summed over the perturbed images for the entire MNIST dataset as a function of t 1 and t 2 . Again, the value at (0, 0) location is the loss of all (MNIST) clean images. The same visualization approach is taken in FIG3 with the difference that the two search directions are defined by the adversary found by the IFGSM attack for each clean image. In both figures, it can be observed that the loss surface of the e2SAD model is the flattest one with the lowest average value within the large 2-dimensional adversarial searching space. This is consistent with the empirically observed effectiveness of e2SAD's defense against both FGSM and IFGSM attacks.
We proposed a time-efficient defense method against one-step and iterative adversarial attacks. Propose a novel, computationaly efficient method named e2SAD which generates sets of two training adversarial samples for each clean training sample. The paper introduces a two-step adversarial defense method, to generate two adversarial examples per clean sample and include them in the actual training loop to achieve robustness and claiming it can outperform more expensive iterative methods. The paper presents a 2-step approach to generate strong adversarial examples at a far lesser cost as compared to recent iterative multi-step adversarial attacks.