Deep neural networks (DNNs) are widely adopted in real-world cognitive applications because of their high accuracy. The robustness of DNN models, however, has been recently challenged by adversarial attacks where small disturbance on input samples may result in misclassification. State-of-the-art defending algorithms, such as adversarial training or robust optimization, improve DNNs' resilience to adversarial attacks by paying high computational costs. Moreover, these approaches are usually designed to defend one or a few known attacking techniques only. The effectiveness to defend other types of attacking methods, especially those that have not yet been discovered or explored, cannot be guaranteed. This work aims for a general approach of enhancing the robustness of DNN models under adversarial attacks. In particular, we propose Bamboo -- the first data augmentation method designed for improving the general robustness of DNN without any hypothesis on the attacking algorithms. Bamboo augments the training data set with a small amount of data uniformly sampled on a fixed radius ball around each training data and hence, effectively increase the distance between natural data points and decision boundary. Our experiments show that Bamboo substantially improve the general robustness against arbitrary types of attacks and noises, achieving better results comparing to previous adversarial training methods, robust optimization methods and other data augmentation methods with the same amount of data points. In recent years, thanks to the availability of large amounts of training data, deep neural network (DNN) models (e.g., convolutional neural networks (CNNs)) have been widely used in many realworld applications such as handwritten digit recognition ), large-scale object classification BID18 ), human face identification BID2 ) and complex control problems BID12 ). Although DNN models have achieved close to or even beyond human performance in many applications, they exposed a high sensitivity to input data samples and therefore are vulnerable to the relevant attacks. For example, adversarial attacks apply a "small" perturbation on input samples, which is visually indistinguishable by humans but can result in the misclassification of DNN models. Several attacking algorithms have been also proposed, including FGSM BID20 ), DeepFool ), CW BID0 ) and PGD BID11 ) etc., indicating a serious threat against the systems using DNN models.Many approaches have also been proposed to defend against adversarial attacks. adversarial training, for example, adds the classification loss of certain known adversarial examples into the total training loss function: BID5 use the FGSM noise for adversarial training and BID11 use the PGD noise as the adversaries. These approaches can effectively improve the model's robustness against a particular attacking algorithm, but won't guarantee the performance against other kinds of attacks BID0 ). Optimization based methods take the training process as a min-max problem and minimize the loss of the worst possible adversarial examples, such as what were done by BID19 and BID21 . The approach can increase the margin between training data points and the decision boundary along some directions. However, solving the min-max problem on-the-fly generates a high demand for the computational load. For large models like VGG BID18 ) and ResNet BID6 ), optimizing the min-max problem could be extremely difficult. A large gap exists between previously proposed algorithms aiming for defending against adversarial attacks and the goal of efficiently improving the overall robustness of DNN models without any hypothesis on the attacking algorithms.Generally speaking, defending against adversarial attacks can be considered as a special case of increasing the generalizability of machine learning models to unseen data points. Data augmentation method, which is originally proposed for improving the model generalizability, may also be effective to improve the DNN robustness against adversarial attacks. Previous studies show that augmenting the original training set with shifted or rotated version of the original data can make the trained classifier robust to shift and rotate transformations of the input BID17 ). Training with additional data sampled from a Gaussian distribution centered at the original training data can also effectively enahnce the model robustness against natural noise BID1 ). The recently proposed Mixup method BID23 ) augmented the training set with linear combinations of the original training data and surprisingly improved the DNN robustness against adversarial attacks. Although these data augmentation methods inspired our work, they may not offer the most efficient way to enhance the adversarial robustness of DNN as they are not designated to defend adversarial attacks.In this work, we propose Bamboo-a ball shape data augmentation technique aiming for improving the general robustness of DNN against adversarial attacks from all directions. Bamboo augments the training set with data uniformly sampled on a fixed radius ball around each training data point. Our theoretical analysis shows that without requiring any prior knowledge of the attacking algorithm, training the DNN classifier with our augmented data set can effectively enhance the general robustness of the DNN models against the adversarial noise. Our experiments show that Bamboo offers a significantly enhanced model robustness comparing to previous robust optimization methods, without suffering from the high computational complexity of these prior works. Comparing to other data augmentation method, Bamboo can also achieve further improvement of the model robustness using the same amount of augmented data. Most importantly, as our method makes no prior assumption on the distribution of adversarial examples, it is able to work against all kinds of adversarial and natural noise. To authors' best knowledge, Bamboo is the first data augmentation method specially designed for improving the general robustness of DNN against all directions of adversarial attacks and noise.The remaining of the paper is organized as follows. Section 2 explains how to measure model robustness and summaries previous research on DNN robustness improvement; In Section 3, we elaborate Bamboo's design principle and the corresponding theoretical analysis. Section 4 empirically discusses the parameter selection and performance of our method and compares it with some related works; At the end, we conclude the paper and discuss the future work in Section 5. In this work we propose Bamboo, the first data augmentation method that is specially designed for improving the overall robustness of DNNs. Without making any assumption on the distribution of adversarial examples, Bamboo is able to improve the DNN robustness against attacks from all directions. Previous analysis and experiment results have proven that by augmenting the training set with data points uniformly sampled on a r-radius ball around original training data, Bamboo is able to effectively improve the robustness of DNN models against different kinds of attacks comparing to previous adversarial training or robust optimization methods, and can achieve stable performance on large DNN models or facing strong adversarial attacks. With the same amount of augmented data, Bamboo is able to achieve better performance against adversarial attacks comparing to other data augmentation methods.We have shown that the resulted network robustness improves as we increase the radius of the ball or the number of augmented data points. In future work we will discuss the theoretical relationship between the resulted DNN robustness and the parameters in our method, and how will the change in the scale of the classification problem affect such relationship. We will also propose new training tricks better suited for training with augmented dataset. As we explore these theoretical relationships and training tricks in the future, we will be able to apply our method more effectively on any new DNN models to improve their robustness against any kinds of adversarial attacks.
The first data augmentation method specially designed for improving the general robustness of DNN without any hypothesis on the attacking algorithms. Proposes a data augmentation training method to gain model robustness against adversarial perturbations, by augmenting uniformly random samples from a fixed-radius sphere centered at training data. 