<p>Here an implementation from <a href="https://gist.github.com/wassname/7793e2058c5c9dacb5212c0ac0b18a8a" rel="nofollow noreferrer">Github</a> that would maybe help</p>
<pre><code>def dice_coef(y_true, y_pred, smooth=1):
    &quot;&quot;&quot;
    Dice = (2*|X &amp; Y|)/ (|X|+ |Y|)
         =  2*sum(|A*B|)/(sum(A^2)+sum(B^2))
    ref: https://arxiv.org/pdf/1606.04797v1.pdf
    &quot;&quot;&quot;
    intersection = K.sum(K.abs(y_true * y_pred), axis=-1)
    return (2. * intersection + smooth) / (K.sum(K.square(y_true),-1) + K.sum(K.square(y_pred),-1) + smooth)

def dice_coef_loss(y_true, y_pred):
    return 1-dice_coef(y_true, y_pred)
</code></pre>
TensorFlow: What is wrong with my (generalized) dice loss implementation?
<p>I use TensorFlow 1.12 for semantic (image) segmentation based on materials. With a multinomial cross-entropy loss function, this yields okay-ish results, especially considering the sparse amount of training data I´m working with, with mIoU of 0.44:</p>

<p><a href="https://i.stack.imgur.com/7kTAi.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/7kTAi.jpg" alt="Segmentations after training with cross entropy loss"></a></p>

<p>When I replace this with my dice loss implementation, however, the networks predicts way less smaller segmentations, which is contrary to my understanding of its theory. I thought it´s supposed to work better with imbalanced datasets and should be better at predicting the smaller classes:</p>

<p><a href="https://i.stack.imgur.com/SSQRX.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/SSQRX.jpg" alt="Segmentations after training with dice loss"></a></p>

<p>A table visualizes this better; as you can see, with dice loss a lot more smaller classes are never predicted (hence the undefined precision). With cross-entropy, at least some predictions are made for all classes:</p>

<p><a href="https://i.stack.imgur.com/O5XOk.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/O5XOk.jpg" alt="Table comparing metrics for the different losses"></a></p>

<p>I initially thought that this is the networks way of increasing mIoU (since my understanding is that dice loss optimizes dice loss directly). However, mIoU with dice loss is 0.33 compared to cross entropy´s 0.44 mIoU, so it has failed in that regard. I´m now wondering whether my implementation is correct:</p>

<pre><code>def dice_loss(onehots_true, logits):
    probabilities = tf.nn.softmax(logits)
    #weights = 1.0 / ((tf.reduce_sum(onehots_true, axis=0)**2) + 1e-3)
    #weights = tf.clip_by_value(weights, 1e-17, 1.0 - 1e-7)
    numerator = tf.reduce_sum(onehots_true * probabilities, axis=0)
    #numerator = tf.reduce_sum(weights * numerator)
    denominator = tf.reduce_sum(onehots_true + probabilities, axis=0)
    #denominator = tf.reduce_sum(weights * denominator)
    loss = 1.0 - 2.0 * (numerator + 1) / (denominator + 1)
    return loss
</code></pre>

<p>Some implementations I found use weights, though I am not sure why, since mIoU isn´t weighted either. At any rate, training is prematurely stopped after one a few epochs with dreadful test results when I use weights, hence I commented them out.</p>

<p>Does anyone see anything wrong with my dice loss implementation? I pretty faithfully followed online examples.</p>

<p>In order to speed up the labeling process, I only annotated with parallelogram shaped polygons, and I copied some annotations from a larger dataset. This resulted in only a couple of ground truth segmentations per image:</p>

<p><a href="https://i.stack.imgur.com/9ExFT.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/9ExFT.jpg" alt="Ground truth annotations"></a></p>

<p>(This image actually contains slightly more annotations than average.)</p>