Deep neural networks provide state-of-the-art performance for many applications of interest. Unfortunately they are known to be vulnerable to adversarial examples, formed by applying small but malicious perturbations to the original inputs. Moreover, the perturbations can transfer across models: adversarial examples generated for a specific model will often mislead other unseen models. Consequently  the adversary can leverage it to attack against the deployed black-box systems. 
 In this work, we demonstrate that the adversarial perturbation can be decomposed into two components: model-specific and data-dependent one, and it is the latter that mainly contributes to the transferability. Motivated by this understanding, we propose to craft adversarial examples by utilizing the noise reduced gradient (NRG) which approximates the data-dependent component. Experiments on various classification models trained on ImageNet demonstrates that the new approach enhances the transferability dramatically. We also find that low-capacity models have more powerful attack capability than high-capacity counterparts, under the condition that they have comparable test performance.   These insights give rise to a principled manner to construct adversarial examples with high success rates and could potentially provide us guidance for designing effective defense approaches against black-box attacks. With the resurgence of neural networks, more and more large neural network models are applied in real-world applications, such as speech recognition, computer vision, etc. While these models have exhibited good performance, recent works BID15 ; BID5 show that an adversary is always able to fool the model into producing incorrect outputs by manipulating the inputs maliciously. The corresponding manipulated samples are called adversarial examples. However, how to understand this phenomenon BID5 ; BID17 ) and how to defend against adversarial examples effectively BID6 ; BID16 ; BID3 ) are still open questions. Meanwhile it is found that adversarial examples can transfer across different models, i.e., the adversarial examples generated from one model can also fool another model with a high probability. We refer to such property as transferability, which can be leveraged to attack black-box systems BID13 ; BID8 ).The phenomenon of adversarial vulnerability was first introduced and studied in BID15 . The authors modeled the adversarial example generation as an optimization problem solved by box-constraint L-BFGS, and also attributed the presence of adversarial examples to the strong nonlinearity of deep neural networks. BID5 argued instead that the primary cause of the adversarial instability is the linear nature and the high dimensionality, and the view yielded the fast gradient sign method (FGSM) . Similarly based on an iterative linearization of the classifier, BID11 proposed the DeepFool method. In BID6 ; BID16 , it was shown that the iterative gradient sign method provides stronger white-box attacks but does not work well for black-box attacks. BID8 analyzed the transferability of adversarial examples in detail and proposed ensemble-based approaches for effective black-box attacks. In BID3 it was demonstrated that high-confidence adversarial examples that are strongly misclassified by the original model have stronger transferability.In addition to crafting adversarial examples for attacks, there exist lots of works on devising more effective defense. BID12 proposed the defensive distillation. BID5 introduced the adversarial training method, which was examined on ImageNet by BID6 and BID16 . BID9 utilized image transformation, such as rotation, translation, and scaling, etc, to alleviate the harm of the adversarial perturbation. Instead of making the classifier itself more robust, several works BID7 ; BID4 ) attempted to detect the adversarial examples, followed by certain manual processing. Unfortunately, all of them can be easily broken by designing stronger and more robust adversarial examples BID3 ; BID0 ).In this work, we give an explanation for the transferability of adversarial examples and use the insight to enhance black-box attacks. Our key observation is that adversarial perturbation can be decomposed into two components: model-specific and data-dependent one. The model-specific component comes from the model architecture and random initialization, which is noisy and represents the behavior off the data manifold. In contrast, the data-dependent component is smooth and approximates the ground truth on the data manifold. We argue that it is the data-dependent part that mainly contributes to the transferability of adversarial perturbations across different models. Based on this view, we propose to construct adversarial examples by employing the data-dependent component of gradient instead of the gradient itself. Since this component is estimated via noise reduction strategy, we call it noise-reduced gradient (NRG) method. Benchmark on the ImageNet validation set demonstrates that the proposed noise reduced gradient used in conjunction with other known methods could dramatically increase the success rate of black-box attacks. to perform black-box attacks over ImageNet validation set.We also explore the dependence of success rate of black-box attacks on model-specific factors, such as model capacity and accuracy. We demonstrate that models with higher accuracy and lower capacity show stronger capability to attack unseen models. Moreover this phenomenon can be explained by our understanding of transferability, and may provide us some guidances to attack unseen models. In this paper, we have verified that an adversarial perturbation can be decomposed into two components: model-specific and data-dependent ones. And it is the latter that mainly contributes to the transferability of adversarial examples. Based on this understanding, we proposed the noise-reduced gradient (NRG) based methods to craft adversarial examples, which are much more effective than previous methods. We also show that the models with lower capacity and higher test accuracy are endowed with stronger capability for black-box attacks.In the future, we will consider combining NRG-based methods with adversarial training to defend against black-box attacks. The component contributing to the transferability is data-dependent, which is intrinsically low-dimensional, so we hypothesize that black-box attacks can be defensible. On the contrary, the white-box attack origins from the extremely high-dimensional ambient space, thus its defense is much more difficult. Another interesting thread of future research is to learn stable features beneficial for transfer learning by incorporating our NRG strategy, since the reduction of model-specific noise can lead to more accurate information on the data manifold.
We propose a new method for enhancing the transferability of adversarial examples by using the noise-reduced gradient. This paper postulates that an adversarial perturbation consists of a model-specific and data-specific component, and that amplification of the latter is best suited for adversarial attacks. This paper focuses on enhancing the transferability of adversarial examples from one model to another model.