title: In tensorflow/keras, why train_on_batch output loss is different when recomputed after training using predict_on_batch?
body: <p>I created a model using keras and I trained it using <code>train_on_batch</code>. To check if the model does what it's supposed to do, I recomputed the loss before and after the training phase using the <code>predict_on_batch</code> method. But, as you'd guess when reading the title, I don't have the same output losses.</p>
  
  <p>Here follows a basic code to illustrate my problem:</p>
  
  <pre><code>from tensorflow.keras.models import Sequential
  from tensorflow.keras.layers import Dense
  import tensorflow as tf
  import numpy as np 
  
  # Loss definition
  def mse(y_true, y_pred):
      return tf.reduce_mean(tf.square(y_true-y_pred))
  
  # Model definition
  model = Sequential()
  model.add(Dense(1))
  model.compile('rmsprop',mse)
  
  # Data creation
  batch_size = 10
  x = np.random.random_sample([batch_size,10])
  y = np.random.random_sample(batch_size)
  
  # Print loss before training
  y_pred = model.predict_on_batch(x)
  print("Before: " + str(mse(y,y_pred).numpy()))
  
  # Print loss output from train_on_batch
  print("Train output: " + str(model.train_on_batch(x,y)))
  
  # Print loss after training
  y_pred = model.predict_on_batch(x)
  print("After: " + str(mse(y,y_pred).numpy()))
  </code></pre>
  
  <p>With this code I get the following outputs:</p>
  
  <pre><code>Before: 0.28556848
  Train output: 0.29771945
  After: 0.27345362
  </code></pre>
  
  <p>I'd suppose that the training loss and the loss computed after training should be the same. So I'd like to understand why not?</p>