Uncertainty Aware Neural Network from Similarity and Sensitivity
Researchers have proposed several approaches for neural network (NN) based
uncertainty quantification (UQ). However, most of the approaches are developed
considering strong assumptions. Uncertainty quantification algorithms often
perform poorly in an input domain and the reason for poor performance remains
unknown. Therefore, we present a neural network training method that considers
similar samples with sensitivity awareness in this paper. In the proposed NN
training method for UQ, first, we train a shallow NN for the point prediction.
Then, we compute the absolute differences between prediction and targets and
train another NN for predicting those absolute differences or absolute errors.
Domains with high average absolute errors represent a high uncertainty. In the
next step, we select each sample in the training set one by one and compute
both prediction and error sensitivities. Then we select similar samples with
sensitivity consideration and save indexes of similar samples. The ranges of an
input parameter become narrower when the output is highly sensitive to that
parameter. After that, we construct initial uncertainty bounds (UB) by
considering the distribution of sensitivity aware similar samples. Prediction
intervals (PIs) from initial uncertainty bounds are larger and cover more
samples than required. Therefore, we train bound correction NN. As following
all the steps for finding UB for each sample requires a lot of computation and
memory access, we train a UB computation NN. The UB computation NN takes an
input sample and provides an uncertainty bound. The UB computation NN is the
final product of the proposed approach. Scripts of the proposed method are
available in the following GitHub repository: github.com/dipuk0506/UQ
H M Dipu Kabir, Subrota Kumar Mondal, Sadia Khanam, Abbas Khosravi, Shafin Rahman, Mohammad Reza Chalak Qazani, Roohallah Alizadehsani, Houshyar Asadi, Shady Mohamed, Saeid Nahavandi, U Rajendra Acharya
# Uncertainty Aware Neural Network from Similarity and Sensitivity

###### Abstract

Researchers have proposed several approaches for neural network (NN) based uncertainty quantification (UQ). However, most of the approaches are developed considering strong assumptions. Uncertainty quantification algorithms often perform poorly in an input domain and the reason for poor performance remains unknown. Therefore, we present a neural network training method that considers similar samples with sensitivity awareness in this paper. In the proposed NN training method for UQ, first, we train a shallow NN for the point prediction. Then, we compute the absolute differences between prediction and targets and train another NN for predicting those absolute differences or absolute errors. Domains with high average absolute errors represent a high uncertainty. In the next step, we select each sample in the training set one by one and compute both prediction and error sensitivities. Then we select similar samples with sensitivity consideration and save indexes of similar samples. The ranges of an input parameter become narrower when the output is highly sensitive to that parameter. After that, we construct initial uncertainty bounds (UB) by considering the distribution of sensitivity aware similar samples. Prediction intervals (PIs) from initial uncertainty bounds are larger and cover more samples than required. Therefore, we train bound correction NN. As following all the steps for finding UB for each sample requires a lot of computation and memory access, we train a UB computation NN. The UB computation NN takes an input sample and provides an uncertainty bound. The UB computation NN is the final product of the proposed approach. Scripts of the proposed method are available in the following GitHub repository: [https://github.com/dipuk0506/UQ](https://github.com/dipuk0506/UQ)

keywords: Uncertainty Bound, Probabilistic Forecast, Neural Network, Prediction Interval, Uncertainty Quantification, Heteroscedastic Uncertainty. +
Footnote †: journal:

## 1 Introduction

Whenever any prediction system receives an unexpectedly high prediction error, people try to investigate the reason for that failure. NNs consist of weights and biases. Values of those parameters are determined during the training. Therefore, the training procedure and the sample distribution play a vital role in NNs heteroscedastic performance [1; 2]. The knowledge of sample distribution also helps individuals in the investigation. Moreover, NNs training needs to be robust to achieve competitive performance on rare and critical samples, and a high overall performance [3; 4].

Traditional regressive models can find the regression mean of a quantity for any input combination within the range [5; 6]. They also provide an overall statistical error, such as the mean-square-error (MSE), the root-mean-square-error (RMSE), etc [7]. Traditional point prediction models, with an overall statistical error, cannot represent the level of heteroscedastic uncertainty. The level of uncertainty can be high in one input domain and low in another input domain [8; 9]. Prediction intervals with a coverage probability can indicate the level of heteroscedastic uncertainty. Regions with narrower intervals have lower uncertainty. Regions with wider interval have lower uncertainty [10]. Uncertainty quantification is becoming popular in various fields [11; 12]. Neural networks are also getting popularity due to their optimal performances [13]. There exist several popular approaches for constructing prediction intervals. Bayesian neural networks are getting popular due to their applications in deep learning. However, Bayesian regressive neural networks struggle to maintain a coverage probability, close to the expected coverage probability. The cost-function-based prediction intervals can provide a narrow interval and the coverage probability becomes very close to the expected coverage probability [14]. However, there are different cost functions proposed by different researchers, and there exist debates on their acceptability. A group of researchers prescribed reducing failure distance and some other researchers prescribed bringing the target near the mid-interval. There is also debate on penalizing high coverage probability. Therefore, we propose a method that trains NNs based on similar samples instead of a cost function.

Although neural networks have brought promising results