title: Gradually update weights of custom loss in Keras during training
body: <p>I defined a custom loss function in Keras (tensorflow backend) that is comprised of reconstruction MSE and the kullback leibler divergence between the learned probability distribution and a standard normal distribution.  (It is for a variational autoencoder.)</p>
  
  <p>I want to be able to slowly increase how much the cost is affected by the KL divergence term during training, with a weight called "reg", starting at reg=0.0 and increasing until it gets to 1.0. I would like the rate of increase to be tuned as a hyperparameter.(As of now, I just have the "reg" parameter set constant at 0.5.)</p>
  
  <p>Is there functionality in Keras to do this?</p>
  
  <pre><code>def vae_loss(y_true,y_pred):
      reg = 0.5
      # Average cosine distance for all words in a sequence 
      reconstruction_loss = tf.reduce_mean(mean_squared_error(y_true, y_pred),1)    
      # Second part of the loss ensures the z probability distribution doesn't stray too far from normal
      KL_divergence_loss = tf.reduce_mean(tf.log(z_sigma) + tf.div((1 + tf.square(z_mu)),2*tf.square(z_sigma)) - 0.5,1)
      loss = reconstruction_loss + tf.multiply(reg,KL_divergence_loss)
      return loss
  </code></pre>