title: Post-process multi-class predictions for image segmentation?
body: <p>My FCN is trained to detect 10 different classes and produces an output of <code>500x500x10</code> with each of the final dimensions being the prediction probabilities for a different class. </p>
  
  <p>Usually, I've seen using a uniform threshold, for instance <code>0.5</code>, to binarize the probability matrices. However, in my case, this doesn't quite cut it because the IoU for some of the classes increases when the threshold is <code>0.3</code> and for other classes it is <code>0.8</code>. </p>
  
  <p>Hence, I don't have to arbitrarily pick the threshold for each class but rather use a more probabilistic approach to finalizing the threshold values. I thought of using CRFs but this also requires the thresholding to have already been done. Any ideas on how to proceed?</p>
  
  <p>Example: consider an image of a forest with 5 different birds. Now im trying to output an image that has segmented the forest and the five birds, 6 classes, each with a separate label. The network outputs 6 confusion matrices indicating the confidence that a pixel falls into a particular class. Now, the correct answer for a pixel isnt always the class  with the highest confidence value. Therefore, a one size fits all method or a max value method won't work. </p>