<p>I suggest starting with a base architecture used in practice like this one in nerve-segmentation: <a href="https://github.com/EdwardTyantov/ultrasound-nerve-segmentation" rel="nofollow noreferrer">https://github.com/EdwardTyantov/ultrasound-nerve-segmentation</a>. Here a dice_loss is used as a loss function. This works very well for a two class problem as has been shown in literature: <a href="https://arxiv.org/pdf/1608.04117.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1608.04117.pdf</a>.</p>

<p>Another loss function that has been widely used is cross entropy for such a problem. For problems like yours most commonly long and short skip connections are deployed to stabilize training as denoted in the paper above.  </p>
Semantic Segmentation with Encoder-Decoder CNNs
<p>Appologizes for misuse of technical terms.
I am working on a project of semantic segmentation via CNNs ; trying to implement an architecture of type Encoder-Decoder, therefore output is the same size as the input.</p>

<p>How do you design the labels ? 
What loss function should one apply ? Especially in the situation of heavy class inbalance (but the ratio between the classes is variable from image to image). </p>

<p>The problem deals with two classes (objects of interest and background). I am using Keras with tensorflow backend.</p>

<p>So far, I am going with designing expected outputs to be the same dimensions as the input images, applying pixel-wise labeling. Final layer of model has either softmax activation (for 2 classes), or sigmoid activation ( to express probability that the pixels belong to the objects class). I am having trouble with designing a suitable objective function for such a task, of type:</p>

<p>function(y_pred,y_true),  </p>

<p>in agreement with Keras. </p>

<p>Please,try to be specific with the dimensions of tensors involved (input/output of the model). Any thoughts and suggestions are much appreciated. Thank you !</p>