title: Keras validation loss
body: <p>In training keras/tensorflow networks I observe losses in validation datasets much lower than in training datasets - by orders of magnitude. I am using the adam optimizer and mean_squared_error loss. The behavior is consistent over different network types and dataset sizes. I would expect mean loss in validation to greater than in train. Perhaps mean_squared_error is erroneously absolute squared error, which could account for lower values since validation is typically smaller than train? If not then what explains this behavior?</p>