<p>I think you are gonna think about Bayesian Learning. First, talking about <strong>uncertainty</strong>.</p>

<p>For example, given several pictures of dog breeds as training data—when a user uploads a photo of his dog—the hypothetical website should return a prediction with rather high confidence. But what should happen if a user uploads a photo of a cat and asks the website to decide on a dog breed?</p>

<p>The above is an example of out of distribution test data. The model has been trained on photos of dogs of different breeds, and has (hopefully) learnt to distinguish between them well. But the model has never seen a cat before, and a photo of a cat would lie outside of the data distribution the model was trained on. This illustrative example can be extended to more serious settings, such as MRI scans with structures a diagnostics system has never observed before, or scenes an autonomous car steering system has never been trained on.</p>

<p>A possible desired behaviour of a model in such cases would be to return a prediction (attempting to extrapolate far away from our observed data), but return an answer with the added information that the point lies outside of the data distribution. We want our model to possess some quantity conveying a high level of <strong>uncertainty</strong> with such inputs (alternatively, conveying low confidence).</p>

<p>Then, I think you could read briefly <a href="https://arxiv.org/abs/1506.02142" rel="nofollow noreferrer">this paper</a> when they also apply to classification task and generate uncertainty for classes (dog, cat...). From this paper, you can extend your finding to application using this paper, and I think you will find what you want.</p>
CNN multi-class network
<p>what approach should i take when I want my CNN multi-class network to output something like <code>[0.1, 0,1]</code> when image doesn't belong
to any class. Using softmax and categorical_crossentropy for multi-class would give me output that sums up to 1 so still not what I want.
I'm new to neural networks so sorry for silly question and thanks in advance for any help.</p>