Recent research has shown that CNNs are often overly sensitive to high-frequency textural patterns. Inspired by the intuition that humans are more sensitive to the lower-frequency (larger-scale) patterns we design a regularization scheme that penalizes large differences between adjacent components within each convolutional kernel. We apply our regularization onto several popular training methods, demonstrating that the models with the proposed smooth kernels enjoy improved adversarial robustness. Further, building on recent work establishing connections between adversarial robustness and interpretability, we show that our method appears to give more perceptually-aligned gradients. In recent years, deep learning models have demonstrated remarkable capabilities for predictive modeling in computer vision, leading some to liken their abilities on perception tasks to those of humans (e.g., Weyand et al., 2016) . However, under closer inspection, the limits of such claims to the narrow scope of i.i.d. data become clear. For example, when faced with adversarial examples (Szegedy et al., 2013; Goodfellow et al., 2015) or even in non-adversarial domain-agnostic cross-domain evaluations (Wang et al., 2019a; b; Carlucci et al., 2019) , performance collapses, dispelling claims of human-like perceptive capabilities and calling into doubt more ambitious applications of this technology in the wild. A long line of recent research has investigated the robustness of neural networks, including investigations of the high-dimension nature of models (Fawzi et al., 2018) , enlarging the gaps between decision boundaries (Zhang et al., 2019a) , training the models with augmented examples through attack methods (Madry et al., 2018) , and even guaranteeing the robustness of models within given radii of perturbation (Wong & Kolter, 2018; Cohen et al., 2019) . Compared to earlier methods, these recent works enjoy stronger robustness both as assessed via theoretical guarantees and empirically via quantitative performance against strong attacks. However, despite the success of these techniques, vulnerabilities to new varieties of attacks are frequently discovered (Zhang et al., 2019b) . In this paper, we aim to lessen the dependency of neural networks on high-frequency patterns in images, regularizing CNNs to focus on the low-frequency components. Therefore, the main argument of this paper is that: by regularizing the CNN to be most sensitive to the low-frequency components of an image, we can improve the robustness of models. Interestingly, this also appears to lead to more perceptually-aligned gradients. Further, as Wang et al. (2019c) explicitly defined the low (or high)-frequency components as images reconstructed from the low (or high)-end of the image frequency domain (as is frequently discussed in neuroscience literature addressing human recognition of shape (Bar, 2004) or face (Awasthi et al., 2011) ), we continue with this definition and demonstrate that a smooth kernel can filter out the high-frequency components and improve the models' robustness. We test our ideas and show the empirical improvement over popular adversarial robust methods with standard evaluations and further use model interpretation methods to understand how the models make decisions and demonstrate that the regularization helps the model to generate more perceptually-aligned gradients. Inspired by neuroscience literature emphasizing the connection between low-frequency components and shape recognition (Bar, 2004; Awasthi et al., 2011) , we proposed a smooth kernel regularization that forces the CNN to learn smooth convolutional kernels (kernels with small differences among adjacent weights) during training. As the relation between smoothness and low-frequency can be argued intuitively and supported by some known results in proved theorems (Titchmarsh, 1948; Bracewell, 1986; Platonov, 2005) , our regularization should help the model to depend more on the low-frequency components of images. To verify the effectiveness of the regularization, we plug in the idea onto multiple training losses, including the vanilla loss, Trades loss (Zhang et al., 2019a) , the adversarial training loss (Madry et al., 2018) , as well as a variation of Logit Pairing loss (Kannan et al., 2018
We introduce a smoothness regularization for convolutional kernels of CNN that can help improve adversarial robustness and lead to perceptually-aligned gradients This paper proposes a new regularization scheme that encourages convolutional kernels to be smoother, arguing that reducing neural network reliance on high-frequency components helps robustness against adversarial examples.  The authors propose a method for learning smoother convolutional kernels, specifically, a regularizer penalizing large changes between consecutive pixels of the kernel with the intuition of penalizing the use of high-frequency input components.