title: What is the replace for softmax layer in case more than one output can be activated?
body: <p>For example, I have CNN which tries to predict numbers from MNIST dataset (code written using Keras). It has 10 outputs, which form softmax layer. Only one of outputs can be true (independently for each digit from 0 to 9):</p>
  
  <pre><code>Real: [0, 1, 0, 0, 0, 0, 0, 0, 0, 0]
  Predicted: [0.02, 0.9, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01]
  </code></pre>
  
  <p>Sum of predicted is equal to 1.0 due to definition of softmax.</p>
  
  <p>Let's say I have a task where I need to classify some objects that can fall in several categories:</p>
  
  <pre><code>Real: [0, 1, 0, 1, 0, 1, 0, 0, 0, 1]
  </code></pre>
  
  <p>So I need to normalize in some other way. I need function which gives value on range [0, 1] and which sum can be larger than 1.</p>
  
  <p>I need something like that:</p>
  
  <pre><code>Predicted: [0.1, 0.9, 0.05, 0.9, 0.01, 0.8, 0.1, 0.01, 0.2, 0.9]
  </code></pre>
  
  <p>Each number is probability that object falls in given category. After that I can use some threshold like 0.5 to distinguish categories in which given object falls.</p>
  
  <p>The following questions appear:</p>
  
  <ol>
  <li>So which activation function can be used for this?</li>
  <li>May be this function already exists in Keras? </li>
  <li>May be you can propose some other way to predict in this case?</li>
  </ol>