<p>Read <em>Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He and Piotr Dollár</em> <a href="https://arxiv.org/abs/1708.02002" rel="nofollow noreferrer"><strong>Focal Loss for Dense Object Detection</strong></a> (ICCV 2017). They discuss in length the shortcoming of CE loss when classes are unbalanced and argue (quite compellingly) that WCE simply does not address this limitation of CE.</p>
<p>CE loss never goes to zero: it always has non-zero gradients even if the prediction is perfect. CE strives to increase the <em>margin</em> between the different classes. As a result, when there is an imbalance between classes, CE will put an equal effort into being &quot;more certain&quot; about the dominant class as well as making fewer mistakes on the minority class. Putting weights on the CE will not make a fundamental change to this behavior.<br />
In contrast, what you actually want from a loss function, in this case, is to ignore samples that you already predict correctly, and make an effort to correct wrong predictions. This is usually achieved via hard-negative mining of Focal loss.</p>
Why most works on Cityscapes don't use weighted cross-entropy?
<p>Weight Cross-Entroy (WCE) helps to handle an imbalanced dataset, and Cityscapes is quite imbalanced as seen below:</p>
<p><a href="https://i.stack.imgur.com/ciUzX.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ciUzX.png" alt="enter image description here" /></a></p>
<p>If we check the <a href="https://paperswithcode.com/sota/semantic-segmentation-on-cityscapes" rel="nofollow noreferrer">best benchmarks</a> on this dataset, most of the works use bare CE as a loss function. I don't get it if there are any special causes that would lead WCE to a worse result for <strong>semantic segmentation</strong> tasks on the <strong>mIoU</strong> evaluation.</p>
<p>I'm especially asking because I'm working in an even higher unbalanced dataset (multi-minority classes on the ratio of 1:1000 to the majority classes) and got very surprised when bare CE outperformed WCE on the mIoU metric.</p>
<p>I found so far that WCE can yield many false positives from minority classes, but beyond that, would there be more reasons for it?</p>