<p>When using dropout to estimate the uncertainty (or any other stochastic regularization method), make sure to also checkout our recent work on providing a sampling-free approximation of Monte-Carlo dropout.</p>

<p><a href="https://arxiv.org/pdf/1908.00598.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1908.00598.pdf</a></p>

<p>We essentially follow ur idea. Treat the activations as random variables and then propagate mean and variance using error propagation to the output layer. Consequently, we obtain <strong>two outputs - the mean and the variance</strong>.</p>
How can I predict the expected value and the variance simultaneously with a neural network?
<p>I'd like to use a neural network to predict a scalar value which is the sum of a function of the input values and a random value (I'm assuming gaussian distribution) whose variance also depends on the input values. Now I'd like to have a neural network that has two outputs - the first output should approximate the deterministic part - the function, and the second output should approximate the variance of the random part, depending on the input values. What loss function do I need to train such a network?</p>

<p>(It would be nice if there was an example with Python for Tensorflow, but I'm also interested in general answers. I'm also not quite clear how I could write something like in Python code - none of the examples I found so far show how to address individual outputs from the loss function.)</p>