title: Tf 2.0 : RuntimeError: GradientTape.gradient can only be called once on non-persistent tapes
body: <p>In tf 2.0 DC Gan example in <a href="https://www.tensorflow.org/alpha/tutorials/generative/dcgan" rel="noreferrer">tensorflow 2.0 guide</a>, there are two gradient tapes . See below.</p>
  
  <pre class="lang-py prettyprint-override"><code>@tf.function
  def train_step(images):
      noise = tf.random.normal([BATCH_SIZE, noise_dim])
  
      with tf.GradientTape() as gen_tape, tf.GradientTape() as disc_tape:
        generated_images = generator(noise, training=True)
  
        real_output = discriminator(images, training=True)
        fake_output = discriminator(generated_images, training=True)
  
        gen_loss = generator_loss(fake_output)
        disc_loss = discriminator_loss(real_output, fake_output)
  
      gradients_of_generator = gen_tape.gradient(gen_loss, generator.trainable_variables)
      gradients_of_discriminator = disc_tape.gradient(disc_loss, discriminator.trainable_variables)
  
      generator_optimizer.apply_gradients(zip(gradients_of_generator, generator.trainable_variables))
      discriminator_optimizer.apply_gradients(zip(gradients_of_discriminator, discriminator.trainable_variables))
  </code></pre>
  
  <p>As you can see clearly that there are two gradient tapes. I was wondering what difference does using a single tape make and changed it to the following</p>
  
  <pre class="lang-py prettyprint-override"><code>@tf.function
  def train_step(images):
      noise = tf.random.normal([BATCH_SIZE, noise_dim])
  
      with tf.GradientTape() as tape:
        generated_images = generator(noise, training=True)
  
        real_output = discriminator(images, training=True)
        fake_output = discriminator(generated_images, training=True)
  
        gen_loss = generator_loss(fake_output)
        disc_loss = discriminator_loss(real_output, fake_output)
  
      gradients_of_generator = tape.gradient(gen_loss, generator.trainable_variables)
      gradients_of_discriminator = tape.gradient(disc_loss, discriminator.trainable_variables)
  
      generator_optimizer.apply_gradients(zip(gradients_of_generator, generator.trainable_variables))
      discriminator_optimizer.apply_gradients(zip(gradients_of_discriminator, discriminator.trainable_variables))
  </code></pre>
  
  <p>This gives me the following error : </p>
  
  <pre><code>RuntimeError: GradientTape.gradient can only be called once on non-persistent tapes.
  </code></pre>
  
  <p>I would like to know why two tapes are necessary.
  As of now the documentation on tf2.0 APIs is scanty. Can anyone explain or point me to the right docs/tutorials? </p>