Uncertainty estimation is an essential step in the evaluation of the robustness for deep learning models in computer vision, especially when applied in risk-sensitive areas. However, most state-of-the-art deep learning models either fail to obtain uncertainty estimation or need significant modification (e.g., formulating a proper Bayesian treatment) to obtain it. None of the previous methods are able to take an arbitrary model off the shelf and generate uncertainty estimation without retraining or redesigning it. To address this gap, we perform the first systematic exploration into training-free uncertainty estimation. 
 We propose three simple and scalable methods to analyze the variance of output from a trained network under tolerable perturbations: infer-transformation, infer-noise, and infer-dropout. They operate solely during inference, without the need to re-train, re-design, or fine-tune the model, as typically required by other state-of-the-art uncertainty estimation methods. Surprisingly, even without involving such perturbations in training, our methods produce comparable or even better uncertainty estimation when compared to other training-required state-of-the-art methods. Last but not least, we demonstrate that the uncertainty from our proposed methods can be used to improve the neural network training.
 Deep learning is already able to achieve excellent or even super-human performance in many tasks (Krizhevsky et al., 2012; He et al., 2015; Silver et al., 2016) . While most previous work in the field has focused on improving accuracy in various tasks, in several risk-sensitive areas such as autonomous driving (Chen et al., 2015) and healthcare (Zhang et al., 2019) , reliability and robustness are arguably more important and interesting than accuracy. Recently, several novel approaches have been proposed to take into account an estimation of uncertainty during training and inference. Some use probabilistic formulations for neural networks (Graves, 2011; Hernández-Lobato & Adams, 2015; Wang et al., 2016; Shekhovtsov & Flach, 2018) and model the distribution over the parameters (weights) and/or the neurons. Such formulations naturally produce distributions over the possible outputs. Others utilize the randomness induced during training and inference (e.g., dropout and ensembling) to obtain an uncertainty estimation (Gal & Ghahramani, 2015; Lakshminarayanan et al., 2017 ). All methods above require specific designs or a special training pipeline in order to involve the uncertainty estimation during training. Unfortunately, there are many cases where such premeditated designs or pipelines cannot be implemented. For example, if one wants to study the uncertainty of trained models released online, retraining is never an option, especially when only a black-box model is provided or the training data is not available. Moreover, most models are deterministic and do not have stochasticity. A straightforward solution is to add dropout layers into proper locations and finetune the model (Gal & Ghahramani, 2016) . However, this is impractical for many state-ofthe-art and published models, especially those trained on large datasets (e.g. ImageNet (Deng et al., 2009) ) with a vast amount of industrial computing resources. In addition, models that have already been distilled, pruned, or binarized fall short of fitting re-training (Han et al., 2015a; Hou et al., 2016) . To fill this gap, we first propose and define the problem of training-free uncertainty estimation: how to obtain an uncertainty estimation of any given model without re-designing, re-training, or fine-tuning it. We focus on two scenarios: black-box uncertainty estimation (BBUE), where one has access to the model only as a black box, and gray-box uncertainty estimation (GBUE), where one has access to intermediate-layer neurons of the model (but not the parameters). To the best of our knowledge, ours is the first systematic exploration into the problem. We propose a set of simple and scalable training-free methods to analyze the variance of output from a trained network. Our main idea is to add a tolerable perturbation into inputs or feature maps during inference. Different from an adversarial perturbation aiming to change the outputs during inference (Madry et al., 2017) , a tolerable perturbation does not dramatically alter the original distribution while allowing generation of multiple diverse outputs that could later be used for uncertainty estimation. The first method, which we call infer-transformation, is to apply transformation that exploits the natural characteristics of a CNN: that it is variant to input transformation such as rotation (Cohen & Welling, 2016) . Transformations have been frequently used for augmentation but rarely evaluated for uncertainty estimation. The second method, infer-noise, is to inject Gaussian noise with a zero-mean and a small standard deviation into intermediate-layer neurons. The third one, which we call infer-dropout is to perform inference-time dropout in a chosen layer. Although at first blush infer-dropout is similar to MC-dropout, where dropout is performed during both training and inference in the same layers, they are different in several aspects: (1) Infer-dropout is involved only during inference. (2) Infer-dropout can be applied to arbitrary layers, even those without dropout training. Surprisingly, we find that even without involving dropout during training, infer-dropout is still comparable to, or even better than, MC-dropout for the purpose of uncertainty estimation. For classification, we note that the softmax output in classification models is naturally a distribution, the entropy of which could be directly used for training-free uncertainty estimation. Hence, using entropy for uncertainty estimation qualifies as a training-free method. We evaluate this method in two classification tasks (see details in Appendix A.1) and find that it already yields satisfactory uncertainty estimation (even more correlated with error compared with MC dropout). Therefore in this paper we focus on regression tasks where output distributions are not readily available. Following the previous work, we evaluate our proposed methods on two regression tasks, monocular depth estimation and single image super resolution, shown in Figure 1 . Our major contributions are thus: 1. We perform, to the best of our knowledge, the first systematic exploration of training-free uncertainty estimation during inference with a post-hoc analysis, given any trained models. 2. We propose simple and scalable methods for regression models, using a tolerable perturbation such as infer-transformation or infer-noise injection to effectively and efficiently estimate uncertainty. 3. Surprisingly, we find that our methods are able to generate a comparable or higher correlation between variance and error than baseline methods, MC dropout and deep ensemble, in both large-scale regression tasks. 4. We demonstrate that the uncertainty from the proposed methods can be used to improve neural network training. In the work, we perform the first systematic exploration into training-free uncertainty estimation. We propose three simple, scalable, and effective methods, namely infer-transformation, infer-noise, and infer-dropout, for uncertainty estimation in both black-box and gray-box cases. Surprisingly, our training-free methods achieve comparable or even better results compared to training-required state-of-the-art methods. Furthermore, we demonstrate adding tolerable perturbations is the key to generating uncertainty maps with high correlation to error maps for all methods we studied. Our future work includes evaluating our methods on distilled, pruned and binarized models, as well as generalizing our methods for more complicated noise/transformation (e.g., non-Gaussian noise arbitrary-angle rotation).
A set of methods to obtain uncertainty estimation of any given model without re-designing, re-training, or to fine-tuning it. Describes several approaches for measuring uncertainty in arbitrary neural networks when there is an absence of distortion during training.