<p>Neural networks use gradient descent, to train: In your high-dimensional parameter space, you always adjust them in the direction of the steepest negative gradient to find a minimum. For that, your loss function has to be differentiable. The rounding function, however, is not <a href="http://mathworld.wolfram.com/NearestIntegerFunction.html" rel="nofollow noreferrer">(image source)</a>:</p>
<p><a href="https://i.stack.imgur.com/9FVYk.gif" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/9FVYk.gif" alt="Src.: http://mathworld.wolfram.com/NearestIntegerFunction.html" /></a></p>
<p>As you can see, die gradient is undefined exactly between two integers, and zero everywhere else. Thus, even if you would define the gradient at the discontinuities manually, your backpropagated gradient would always be zero due to chain rule.</p>
<p>I do not know the exact purpose of your network. However, it might be worth trying to convert your network from a regression (where you predict a continuous number) problem into a classification problem, where you predict a class score for each possible integer instead of rounding.</p>
<p><strong>Update:</strong></p>
<p>If you do masking or segmentation,  the real-valued output will give you sort of a 'probability' (at least when using softmax in last layer) that your pixel or voxel belongs to the region you want to mask. If you  round the result, you loose important detail for training your network. A pixel with a score of 0.4 will be given the same score as one with 0.1. Thus, a change a small weight change will not change the loss of your network and gradient descent will not work. The <a href="https://arxiv.org/pdf/1606.04797.pdf" rel="nofollow noreferrer">original paper</a> introducing dice loss for segmentation, also does not use rounding. If you want to map each pixel to foreground/background for visualizatuion purposes, you should do it after computing the loss.</p>
<p>However, you always have the possiblity to define your own 'gradient', since gradient descent is not the only way to optimize. There are derivative-free optimization techniques. But be careful.</p>
<p>Without trying if it works in practice, this would be my approach, when you really don't want to go without the round function (no guaranty that this will yield sensible results in any way): Using distribution theory, you could define a derivative of the round function, as a sum of the derivatives of many <a href="https://en.wikipedia.org/wiki/Heaviside_step_function#Antiderivative_and_derivative" rel="nofollow noreferrer">heaviside functions</a>, leaving you with a <a href="https://en.wikipedia.org/wiki/Dirac_comb" rel="nofollow noreferrer">dirac comb</a>. If you now replace the delta distributions with normal distributions with a small standard deviation, you get the effect, that the gradient in between integers will lead them in the direction of the nearest integer (with the exeption of exactly between, where the derivative of the normal distribution is 0).</p>
<p><strong>Disclaimer:</strong> I've never seen something like this in use anywhere, and the best solution would be to just abandon the round function, but if you feel like experimenting a bit, you could try this. If anyone, has any arguments, why this is just plainly false, please tell me!</p>
Keras Cost Function Error when trying to round predicted tensor to nearest integer
<p>I am using sigmoid activation in the second-last layer and then resizing using <code>tf.images.resize_images()</code> in the last layer. </p>

<p>The target tensor has a maximum value of 1.0. In the dice error cost function.</p>

<pre><code>def dice(y_true, y_pred):
    return 1.0-dice_coef(y_true, y_pred, 1e-5, 0.5)

def dice_coef(y_true, y_pred, smooth, thresh, axis = [1,2,3]):
    y_pred = K.round(y_pred)
    inse = K.sum(K.dot(y_true, K.transpose(y_pred)), axis=axis)
    l = K.sum(y_pred, axis=axis)
    r = K.sum(y_true, axis=axis)
    hard_dice = (2. * inse + smooth) / (l + r + smooth)
    hard_dice = K.mean(hard_dice)
    return hard_dice
</code></pre>

<p>When I run the code I get the error below. However, the error goes away when I remove <code>K.round(y_pred)</code>. Any idea on how to solve this problem?</p>

<pre><code>loss,acc,err = Final_Model.train_on_batch(Train_image,Label)
File "C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\tensorflow-cpu\lib\site-packages\keras\engine\training.py", line 1761, in train_on_batch
self._make_train_function()
  File "C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\tensorflow-cpu\lib\site-packages\keras\engine\training.py", line 960, in _make_train_function
loss=self.total_loss)
 File "C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\tensorflow-cpu\lib\site-packages\keras\legacy\interfaces.py", line 87, in wrapper
return func(*args, **kwargs)
  File "C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\tensorflow-cpu\lib\site-packages\keras\optimizers.py", line 358, in get_updates
new_a = self.rho * a + (1. - self.rho) * K.square(g)
 File "C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\tensorflow-cpu\lib\site-packages\keras\backend\tensorflow_backend.py", line 1358, in square
return tf.square(x)
 File "C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\tensorflow-cpu\lib\site-packages\tensorflow\python\ops\math_ops.py", line 447, in square
return gen_math_ops.square(x, name=name)
 File "C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\tensorflow-cpu\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 2591, in square
result = _op_def_lib.apply_op("Square", x=x, name=name)
 File "C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\tensorflow-cpu\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 508, in apply_op
(input_name, err))
ValueError: Tried to convert 'x' to a tensor and failed. Error: None values not supported` 
</code></pre>