 Conventional out-of-distribution (OOD) detection schemes based on variational autoencoder or Random Network Distillation (RND) are known to assign lower uncertainty to the OOD data than the target distribution. In this work, we discover that such conventional novelty detection schemes are also vulnerable to the blurred images. Based on the observation, we construct a novel RND-based OOD detector, SVD-RND, that utilizes blurred images during training. Our detector is simple, efficient in test time, and outperforms baseline OOD detectors in various domains. Further results show that SVD-RND learns a better target distribution representation than the baselines. Finally, SVD-RND combined with geometric transform achieves near-perfect detection accuracy in CelebA domain. Out-of distribution (OOD), or novelty detection aims to distinguish samples in unseen distribution from the training distribution. A majority of novelty detection methods focus on noise filtering or representation learning. For example, we train an autoencoder to learn a mapping from the data to the bottleneck layer and use the bottleneck representation or reconstruction error to detect an OOD (Sakruada et al., 2014; Pidhorskyi et al., 2018) . Recently, deep generative models (Kingma et al., 2014; Dinh et al., 2017; Kingma et al., 2018; Schlegl et al., 2017) are widely used for novelty detection due to their ability to model high dimensional data. However, OOD detection performance of deep generative models has been called into question since they have been observed to assign a higher likelihood to the OOD data than the training data (Nalisnick et al., 2019; Choi et al., 2018) . On the other hand, adversarial examples are widely employed to fool the classifier, and training classifiers against adversarial attacks has shown effectiveness in detecting unknown adversarial attacks (Tramer et al., 2018) . In this work, we propose blurred data as the adversarial example. When we test novelty detection models on the blurred data generated by Singular Value Decomposition (SVD), we found that the novelty detection models assign higher confidence to the blurred data than the original data. Motivated by this observation, we employ blurring to prevent the OOD detector from overfitting to low resolution. We propose a new OOD detection model, SVD-RND, which is trained using the idea of Random Network Distillation (RND) (Burda et al., 2019) to discriminate the training data from the blurred image. SVD-RND is evaluated in the hard target to OOD domain where vanilla generative models show nearly 50% detection accuracy, such as CIFAR-10 to SVHN and ImageNet to CIFAR-10 (Nalisnick et al., 2019) . Compared to conventional baselines, SVD-RND shows a significant performance gain from 50% to over 90% in these domains. Moreover, SVD-RND shows improvements over baselines on domains where conventional OOD detection schemes show moderate results, such as CIFAR-10 to LSUN. In this work, a blurred image is introduced as an adversarial example to the deep OOD detection method. SVD-RND is employed for adversarial defense against blurred images. SVD-RND achieves significant performance gain in all target : anomaly domains. Even without the validation OOD data, we can design SVD-RND to outperform conventional OOD detection models. We stress that such performance gain is achieved without external data or additional regularization techniques. Furthermore, experiments on SVD-RND and RND show that the neural network can potentially learn to perform OOD detection, however overfits to blurred data. Understanding this phenomenon will be beneficial to performance of the image-based models. We use RND (Burda et al., 2019) as the base model of our OOD detector. RND consists of the trainable predictor network f , and randomly initialized target network g. The predictor network is trained to minimize the l 2 distance against the target network on training data. We do not update the target network g throughout the training phase.
We propose a novel OOD detector that employ blurred images as adversarial examples . Our model achieve significant OOD detection performance in various domains. This paper presents the idea to use blurred images as regularizing examples to improve out-of-distribution detection performance based on Random Network Distillation. This paper tackles out-of-data distribution by leveraging RND applied to data augmentations by training a model to match the outputs of a random network with an augmentation as input.