title: Printing the loss during TensorFlow training
body: <p>I am looking at the TensorFlow "<a href="https://www.tensorflow.org/versions/r1.1/get_started/mnist/beginners#evaluating_our_model" rel="noreferrer">MNIST For ML Beginners</a>" tutorial, and I want to print out the training loss after every training step.</p>
  
  <p>My training loop currently looks like this:</p>
  
  <pre><code>for i in range(100):
      batch_xs, batch_ys = mnist.train.next_batch(100)
      sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})
  </code></pre>
  
  <p>Now, <code>train_step</code> is defined as:</p>
  
  <pre><code>train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)
  </code></pre>
  
  <p>Where <code>cross_entropy</code> is the loss which I want to print out:</p>
  
  <pre><code>cross_entropy = -tf.reduce_sum(y_ * tf.log(y))
  </code></pre>
  
  <p>One way to print this would be to explicitly compute <code>cross_entropy</code> in the training loop:</p>
  
  <pre><code>for i in range(100):
      batch_xs, batch_ys = mnist.train.next_batch(100)
      cross_entropy = -tf.reduce_sum(y_ * tf.log(y))
      print 'loss = ' + str(cross_entropy)
      sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})
  </code></pre>
  
  <p>I now have two questions regarding this:</p>
  
  <ol>
  <li><p>Given that <code>cross_entropy</code> is already computed during <code>sess.run(train_step, ...)</code>, it seems inefficient to compute it twice, requiring twice the number of forward passes of all the training data. Is there a way to access the value of <code>cross_entropy</code> when it was computed during <code>sess.run(train_step, ...)</code>?</p></li>
  <li><p>How do I even print a <code>tf.Variable</code>? Using <code>str(cross_entropy)</code> gives me an error...</p></li>
  </ol>
  
  <p>Thank you!</p>