<p>Well, apparently François Chollet has made a few changes very recently (5 days ago), including changes in how the kl_loss and reconstruction_loss are computed, see <a href="https://github.com/keras-team/keras-io/commit/e68c2209f2e96da84babd5017d15911fa4d3c7e4#diff-8522b8e0350af836c7852feb030176985f2267bf6cbdec97a87fbac090a135a8" rel="nofollow noreferrer">here</a>.</p>
<p>Having run the previous version (that you can find at the link above), I significantly reduced the difference between the two members of the equation, even reducing with increasing epoch (from epoch 7, the difference is &lt;.2), as compared to your values.</p>
<p>It seems that VAE are subject to reconstruction loss underestimation, which is an ongoing issue, and for that, I encourage you to dig a bit in the litterature, with e.g. <a href="https://arxiv.org/pdf/1910.00698.pdf" rel="nofollow noreferrer">this article </a>(may not be the best one).</p>
<p>Hope that helps! At least it's a step forward.</p>
Variational Autoencoder loss not displayed right?
<p>I have implemented a variational autoencoder with the Keras implementation as an example (<a href="https://keras.io/examples/generative/vae/" rel="nofollow noreferrer">https://keras.io/examples/generative/vae/</a>). When plotting the training loss I noticed that these were not the same as displayed in the console. I also saw that the displayed loss in the console in the Keras example was not right considering total_loss = reconstruction_loss + kl_loss.</p>
<p>Is the displayed loss in the console not the total_loss?</p>
<p>My VAE code:</p>
<pre><code>class Sampling(layers.Layer):
    &quot;&quot;&quot;Uses (z_mean, z_log_var) to sample z, the vector encoding a digit.&quot;&quot;&quot;

    def call(self, inputs):
        z_mean, z_log_var = inputs
        batch = tf.shape(z_mean)[0]
        dim = tf.shape(z_mean)[1]
        epsilon = tf.keras.backend.random_normal(shape=(batch, dim))
        return z_mean + tf.exp(0.5 * z_log_var) * epsilon

latent_dim = 100 

encoder_inputs = keras.Input(shape=(64, 64, 3)) #eigentlich 160
x = layers.Conv2D(32, 4, strides=2, padding=&quot;same&quot;)(encoder_inputs)   
x = layers.LeakyReLU(alpha=0.2)(x)
x = layers.Conv2D(32, 3, strides=1, padding=&quot;same&quot;)(x)   
x = layers.LeakyReLU(alpha=0.2)(x)
x = layers.Conv2D(64, 4,strides=2, padding=&quot;same&quot;)(x)
x = layers.LeakyReLU(alpha=0.2)(x)
x = layers.Conv2D(64, 3,strides=1, padding=&quot;same&quot;)(x)
x = layers.LeakyReLU(alpha=0.2)(x)
x = layers.Conv2D(128, 4,strides=2, padding=&quot;same&quot;)(x)  
x = layers.LeakyReLU(alpha=0.2)(x)
x = layers.Conv2D(64, 3,strides=1, padding=&quot;same&quot;)(x)
x = layers.LeakyReLU(alpha=0.2)(x)
x = layers.Conv2D(32, 3,strides=1, padding=&quot;same&quot;)(x)
x = layers.LeakyReLU(alpha=0.2)(x)
x = layers.Conv2D(100, 8,strides=1, padding=&quot;valid&quot;)(x)        
x = layers.LeakyReLU(alpha=0.2)(x)
x = layers.Flatten()(x)
z_mean = layers.Dense(latent_dim, name=&quot;z_mean&quot;)(x)
z_log_var = layers.Dense(latent_dim, name=&quot;z_log_var&quot;)(x)
z = Sampling()([z_mean, z_log_var])
encoder = keras.Model(encoder_inputs, [z_mean, z_log_var, z], name=&quot;encoder&quot;)
encoder.summary()

latent_inputs = keras.Input(shape=(latent_dim,))
x = layers.Reshape((1, 1, 100))(latent_inputs)
x = layers.Conv2DTranspose(100, 8, strides=1, padding=&quot;valid&quot;)(x)
x = layers.LeakyReLU(alpha=0.2)(x)
x = layers.Conv2DTranspose(32, 3, strides=1, padding=&quot;same&quot;)(x)
x = layers.LeakyReLU(alpha=0.2)(x)
x = layers.Conv2DTranspose(64, 3, strides=1, padding=&quot;same&quot;)(x)
x = layers.LeakyReLU(alpha=0.2)(x)
x = layers.Conv2DTranspose(128, 4, strides=2, padding=&quot;same&quot;)(x)
x = layers.LeakyReLU(alpha=0.2)(x)
x = layers.Conv2DTranspose(64, 3, strides=1, padding=&quot;same&quot;)(x)
x = layers.LeakyReLU(alpha=0.2)(x)
x = layers.Conv2DTranspose(64, 4, strides=2, padding=&quot;same&quot;)(x)
x = layers.LeakyReLU(alpha=0.2)(x)
x = layers.Conv2DTranspose(32, 3, strides=1, padding=&quot;same&quot;)(x)
x = layers.LeakyReLU(alpha=0.2)(x)
x = layers.Conv2DTranspose(32, 4, strides=2, padding=&quot;same&quot;)(x)
x = layers.LeakyReLU(alpha=0.2)(x)

decoder_outputs = layers.Conv2DTranspose(3, 3, activation=&quot;sigmoid&quot;, padding=&quot;same&quot;)(x)
decoder = keras.Model(latent_inputs, decoder_outputs, name=&quot;decoder&quot;)
decoder.summary()

class VAE(keras.Model):
    def __init__(self, encoder, decoder, encoder_t1, encoder_t2, encoder_t3, encoder_t4, **kwargs):
        super(VAE, self).__init__(**kwargs)
        self.encoder = encoder
        self.decoder = decoder

    def train_step(self, data):
        if isinstance(data, tuple):
            data = data[0]
        with tf.GradientTape() as tape:
            z_mean, z_log_var, z = encoder(data)
            reconstruction = decoder(z)
            reconstruction_loss = tf.reduce_mean(    #mean
                keras.losses.mse(data, reconstruction)      #binary_crossentropy
            )
            reconstruction_loss *= 64 * 64                                    #entspricht bildgrÃ¶ÃŸe
            kl_loss = 1 + z_log_var - tf.square(z_mean) - tf.exp(z_log_var)
            kl_loss = tf.reduce_mean(kl_loss)      #mean
            kl_loss *= -0.5 
            total_loss = reconstruction_loss + kl_loss
        grads = tape.gradient(total_loss, self.trainable_weights)
        self.optimizer.apply_gradients(zip(grads, self.trainable_weights))
        return {
            &quot;loss&quot;: total_loss,
            &quot;reconstruction_loss&quot;: reconstruction_loss,
            &quot;kl_loss&quot;: kl_loss,
        }
    def call(self, inputs):
      z_mean, z_log_var, z = encoder(inputs)
      reconstruction = decoder(z)
      reconstruction_loss = tf.reduce_mean(
          keras.losses.mse(inputs, reconstruction)
      )
      reconstruction_loss *= 64 * 64
      kl_loss = 1 + z_log_var - tf.square(z_mean) - tf.exp(z_log_var)
      kl_loss = tf.reduce_mean(kl_loss)
      kl_loss *= -0.5 
      total_loss = reconstruction_loss + kl_loss
      self.add_metric(kl_loss, name='kl_loss', aggregation='mean')
      self.add_metric(total_loss, name='total_loss', aggregation='mean')
      self.add_metric(reconstruction_loss, name='reconstruction_loss', aggregation='mean')
      return reconstruction
</code></pre>
<p>When I plot my loss with the following code:</p>
<pre><code>vae_train = vae.fit(
        train_generator,
        steps_per_epoch=nb_train_samples,
        epochs=nb_epoch,
        validation_data=val_generator,
        validation_steps=nb_validation_samples, #141 #3963
        callbacks=[es_callback]
        )

loss = vae_train.history['loss']
val_loss = vae_train.history['val_total_loss']
plt.figure()
plt.plot(range(len(loss)), loss, 'b', label = 'Training loss')
plt.plot(range(len(val_loss)), val_loss, 'm', label='Validation loss')
plt.title('Training and validation loss')
plt.legend()
plt.show()
</code></pre>
<p>The resulting plot displays the loss differently than the displayed loss in the console. As the displayed loss in the console is not reconstruction_loss + kl_loss but the plotted loss is.</p>
<p>For example the displayed loss here is not correct, but it is plotted right: (interestingly the val_total_loss is displayed correctly)</p>
<pre><code>Epoch 20/100
1266/1266 [==============================] - 82s 65ms/step - loss: 45.2503 - reconstruction_loss: 49.9395 - kl_loss: 0.5695 - val_loss: 0.0000e+00 - val_kl_loss: 0.5888 - val_total_loss: 48.9094 - val_reconstruction_loss: 48.3206
</code></pre>