title: How to calculate negative log-likelihoog on MNIST dataset?
body: <p>The following table </p>
  
  <p><a href="https://i.stack.imgur.com/zKdgn.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/zKdgn.png" alt="NLL evaluations on MNIST test set."></a></p>
  
  <p>is from the <a href="https://arxiv.org/pdf/1610.09038.pdf" rel="nofollow noreferrer">Professor Forcing: A New Algorithm for Training Recurrent Networks</a> paper. But I couldn't find their code to calculate NLL. I would like to ask if it is simply the binary cross-entropy. Can I calculate it by using Tensorflow <a href="https://www.tensorflow.org/api_docs/python/tf/nn/sigmoid_cross_entropy_with_logits" rel="nofollow noreferrer">tf.nn.sigmoid_cross_entropy_with_logits</a> function?</p>
  
  <p>In Professor Forcing paper, the evaluation result with teacher forcing is not presented. I trained a straightforward LSTM and achieved an NLL of 80.394. My last question is how likely to get ~80 or ~70?</p>
  
  <p>More specifically, I am trying to generate MNIST images pixel by pixel. My model makes a binary prediction for every pixel, which can take values of 0 and 1. The dimension of both logits and labels are <code>[batch_size, 28*28, 1]</code>, where 28 is the height and width of an MNIST image.</p>