<p>FCN uses per-pixel softmax and a multinominal loss. This means, that the mask prediction task (the boundaries of the object) and the class prediction task (what is the object being masked) are coupled.<br>
Mask-RCNN decouples these tasks: the existing bounding-box prediction (AKA the localization task) head predicts the class, like faster-RCNN, and the mask branch generates a mask <strong>for each class</strong>, without competition among classes (e.g. if you have 21 classes the mask branch predicts 21 masks instead of FCN's single mask with 21 channels). The loss being used is per-pixel sigmoid + binary loss.<br>
Bottom line, it's Sigmoid in Mask-RCNN vs. Soft-max in FCN.<br>
(<a href="https://arxiv.org/pdf/1703.06870.pdf" rel="noreferrer">See table 2.b. in Mask RCNN paper - Ablation section</a>).</p>
What is the loss function of the Mask RCNN?
<p>The paper has clearly mentioned the <strong>classification and regression</strong> losses are identical to the RPN network in the Faster RCNN . Can someone explain the Mask Loss function . How the use FCN to improve ?   </p>