Deep networks face challenges of ensuring their robustness against inputs that cannot be effectively represented by information learned from training data. We attribute this vulnerability to the limitations inherent to activation-based representation. To complement the learned information from activation-based representation, we propose utilizing a gradient-based representation that explicitly focuses on missing information. In addition, we propose a directional constraint on the gradients as an objective during training to improve the characterization of missing information. To validate the effectiveness of the proposed approach, we compare the anomaly detection performance of gradient-based and activation-based representations. We show that the gradient-based representation outperforms the activation-based representation by 0.093 in CIFAR-10 and 0.361 in CURE-TSR datasets in terms of AUROC averaged over all classes. Also, we propose an anomaly detection algorithm that uses the gradient-based representation, denoted as GradCon, and validate its performance on three benchmarking datasets. The proposed method outperforms the majority of the state-of-the-art algorithms in CIFAR-10, MNIST, and fMNIST datasets with an average AUROC of 0.664, 0.973, and 0.934, respectively. The generalizable representation of data from deep network has largely contributed to the success of deep learning in diverse applications (Bengio et al., 2013) . The representation from deep networks is often obtained in the form of activation. The activation is constructed by the weights which contain specific knowledge learned from training samples. Recent studies reveal that deep networks still face robustness issues when input that cannot be properly represented by learned knowledge is given to the networks (Goodfellow et al., 2014; Hendrycks & Dietterich, 2018; Liang et al., 2017) . One of the reasons for the vulnerability of deep networks is the limitation in the activation-based representation, which inherently focused on the learned knowledge. However, the part of the input that causes problems in deep networks is mainly from the information that deep networks were not able to learn from the training data. Therefore, it is more appropriate to complement the representation of input data from the perspective of information that has not been learned for enhancing the robustness of machine learning algorithms. The gradient is another fundamental element in deep networks that is utilized to learn new information from given inputs by updating model weights . It is generated through backpropagation to train deep networks by minimizing designed loss functions (Rumelhart et al., 1986) . During the training of network, the gradient with respect to the weights provides directional information to update the deep network and learn a better representation for the inputs. In other words, gradients guide the network to learn new information that was not learned from data that it has seen so far but is presented in the current input. Considering this role during training, gradients can provide a complementary perspective with respect to activation and characterize missing information that the network has not learned for each unseen image. We demonstrate the role of gradients with an example in Fig. 1 . Assume that a deep network has only learned curved edge features from training images of the digit '0'. During testing, the digit '6' is given to the network. The digit '6' consists of both learned information (curved edges) and missing information (straight edges on top). Since the activation-based representation is constructed based on the information that the network has already learned, the curved part of the digit '6' will be characterized effectively by the activation. However, the network still has to learn the straight edge features to perform successfully on the digit '6'. Therefore, the gradients which guide updates in the deep network can characterize straight edge information that has not been learned. We propose analyzing the representation capability of gradients in characterizing missing information for deep networks. Gradients have been utilized in diverse applications such as adversarial attack generation and visualization (Zeiler & Fergus, 2014; Goodfellow et al., 2014) . However, using gradients with respect to weight as the representation of data has not been actively explored yet. Through the comprehensive analysis with activation-based representations, we show the effectiveness of gradient representation in characterizing the information that has not been learned for deep network. Furthermore, we show that gradient representation can achieve state-of-the-art performance in detecting potentially invalid data for the network. The main contributions of this paper are three folds: i We propose utilizing gradients as a representation to characterize information that has not been learned from the training data but is currently presented in the input data. ii We analyze the representation capability of gradient compared to activation for detecting samples which possess features that have not been learned for the network. iii We propose a gradient-based anomaly detection algorithm that outperforms state-of-the-art algorithms based on activation representations. We propose a gradient-based representation for characterizing information that deep networks have not learned. We introduce our geometric interpretation of gradients and generalize it to high dimensional scenarios of deep learning through the proposed directional constraint on gradients. We also thoroughly evaluate the representation capability of gradients compared to that of activations. We validate the effectiveness of gradients in the context of anomaly detection and show that proposed method based on the gradient representation achieves the state-of-the-art performance in four benchmarking datasets. The experimental results show that the directional information of gradients effectively characterizes diverse missing information by complementing distance information from activations. Also, the gradient-based representation can provide a comprehensive perspective to handle data that cannot be represented by training data in diverse applications aiming to ensure the robustness of deep networks.
We propose a gradient-based representation for characterizing information that deep networks have not learned. The authors present creating representations based on gradients with respect to the weights to supplement information missing from the training dataset for deep networks.