<p>You should.</p>

<p>The way filter weights are initialized there is a hidden assumption that the input signal is roughly zero mean and unit variance. This helps scaling the activations and the gradients.<br>
In practice, you may be able to train a net (for segmentation or any other task for that matter) using the "un-normalized" input images, but it will probably take longer to train and be less stable w.r.t meta parameters such as learning rate and solver type.</p>

<p>For more details see section 2.2 in He et al. <a href="https://arxiv.org/pdf/1502.01852.pdf" rel="nofollow noreferrer">"Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification"</a> (ICCV 2015).</p>
Should I subtract image mean when doing a task of semantic segmentation? why or why not?
<p>Should I subtract image mean or divide 255 for a image on a semantic segmentation task? why or why not?</p>