Recent work has demonstrated that neural networks are vulnerable to adversarial examples, i.e., inputs that are almost indistinguishable from natural data and yet classified incorrectly by the network. To address this problem, we study the adversarial robustness of neural networks through the lens of robust optimization. This approach provides us with a broad and unifying view on much prior work on this topic. Its principled nature also enables us to identify methods for both training and attacking neural networks that are reliable and, in a certain sense, universal. In particular, they specify a concrete security guarantee that would protect against a well-defined class of adversaries. These methods let us train networks with significantly improved resistance to a wide range of adversarial attacks. They also suggest robustness against a first-order adversary as a natural security guarantee. We believe that robustness against such well-defined classes of adversaries is an important stepping stone towards fully resistant deep learning models. Recent breakthroughs in computer vision and speech recognition are bringing trained classifiers into the center of security-critical systems. Important examples include vision for autonomous cars, face recognition, and malware detection. These developments make security aspects of machine learning increasingly important. In particular, resistance to adversarially chosen inputs is becoming a crucial design goal. While trained models tend to be very effective in classifying benign inputs, recent work BID6 BID24 BID8 BID16 BID22 shows that an adversary is often able to manipulate the input so that the model produces an incorrect output.This phenomenon has received particular attention in the context of deep neural networks, and there is now a quickly growing body of work on this topic BID7 BID13 BID20 BID25 BID23 BID27 . Computer vision presents a particularly striking challenge: very small changes to the input image can fool state-of-the-art neural networks with high probability BID24 BID8 BID16 BID22 BID15 . This holds even when the benign example was classified correctly, and the change is imperceptible to a human. Apart from the security implications, this phenomenon also demonstrates that our current models are not learning the underlying concepts in a robust manner. All these findings raise a fundamental question:How can we learn models robust to adversarial inputs?There are now many proposed defense mechanisms for the adversarial setting. Examples include defensive distillation BID18 , feature squeezing BID31 , and several detection approaches for adversarial inputs (see BID4 for references). While these works constitute important first steps in exploring the realm of possibilities, they do not offer a good understanding of the guarantees they provide. We can never be certain that a particular defense mechanism prevents the existence of some well-defined class of adversarial attacks. This makes it difficult to navigate the landscape of adversarial robustness or to fully evaluate the possible security implications. Moreover, subsequent work BID2 BID11 has shown that most of these defenses can be bypassed by stronger, adaptive adversaries.In this paper, we study the adversarial robustness of neural networks through the lens of robust optimization. We use a natural saddle point (min-max) formulation to capture the notion of security against adversarial attacks in a principled manner. This formulation allows us to be precise about the type of security guarantee we would like to achieve, i.e., the broad class of attacks we want to be resistant to (in contrast to defending only against specific known attacks). The formulation also enables us to cast both attacks and defenses into a common theoretical framework. Most prior work on adversarial examples naturally fits into this framework. In particular, adversarial training directly corresponds to optimizing this saddle point problem. Similarly, prior methods for attacking neural networks correspond to specific algorithms for solving the underlying optimization problem.Equipped with this perspective, we make the following contributions.1. We conduct a careful experimental study of the optimization landscape corresponding to this saddle point formulation. Despite the non-convexity and non-concavity of its constituent parts, we find that the underlying optimization problem is tractable after all. In particular, we provide strong evidence that first-order methods can reliably solve this problem and motivate projected gradient descent (PGD) as a universal "first-order adversary", i.e., the strongest attack utilizing the local first order information about the network. We supplement these insights with ideas from real analysis to further motivate adversarial training against a PGD adversary as a strong and natural defense.2. We explore the impact of network architecture on adversarial robustness and find that model capacity plays an important role. To reliably withstand strong adversarial attacks , networks require a significantly larger capacity than for correctly classifying benign examples only. This shows that a robust decision boundary of the saddle point problem can be significantly more complicated than a decision boundary that simply separates the benign data points.3. Building on the above insights, we train networks on MNIST and CIFAR10 that are robust to a wide range of adversarial attacks against adversaries bounded by 0.3 and 8 in ∞ norm respectively. Our approach is based on optimizing the aforementioned saddle point formulation and uses our optimal "first-order adversary". Our best MNIST model achieves an accuracy of more than 89% against the strongest adversaries in our test suite. In particular, our MNIST network is even robust against white box attacks of an iterative adversary. Our CIFAR10 model achieves an accuracy of 46% against the same adversary. Furthermore, in case of the weaker black box (transfer) attacks , our MNIST and CIFAR10 networks achieve an accuracy of more than 95% and 64%, respectively (a more detailed overview can be found in TAB0 ). To the best of our knowledge, we are the first to achieve these levels of robustness on MNIST and CIFAR10 against a broad set of attacks.Overall, these findings suggest that secure neural networks are within reach. In order to further support this claim, we have invited the community to attempt attacks against our MNIST and CIFAR10 networks in the form of an open challenge 1,2 . At the time of writing, we received about fifteen submissions to the MNIST challenge and the best submission achieved roughly 93% accuracy in a black box attack. We received no submissions for the CIFAR10 challenge that went beyond the 64% accuracy of our attack. Considering that other proposed defenses were often quickly broken BID4 , we believe that our robust models are significant progress on the defense side. Furthermore, recent work on verifiable adversarial examples showed that our proposed defense reliably increased the robustness to any ∞ -bounded attack. Our findings provide evidence that deep neural networks can be made resistant to adversarial attacks. As our theory and experiments indicate, we can design reliable adversarial training methods. One of the key insights behind this is the unexpectedly regular structure of the underlying optimization task: even though the relevant problem corresponds to the maximization of a highly non-concave function with many distinct local maxima, their values are highly concentrated. Overall, our findings give us hope that adversarially robust deep learning models may be within current reach.For the MNIST dataset, our networks are very robust, achieving high accuracy for a wide range of powerful adversaries and large perturbations. Our experiments on CIFAR10 have not reached the same level of performance yet. However, our results already show that our techniques lead to significant increase in the robustness of the network. We believe that further exploring this direction will lead to adversarially robust networks for this dataset. We thank Wojciech Matusik for kindly providing us with computing resources to perform this work.
We provide a principled, optimization-based re-look at the notion of adversarial examples, and develop methods that produce models that are adversarially robust against a wide range of adversaries. Investigates a minimax formulation of deep network learning to increase their robustness, using projected gradient descent as the main adversary.  This paper proposes to look at making neural networks resistant to adversarial loss through the framework of saddle-point problems. 