title: Entropy of a MNIST image with Tensorflow
body: <p>I want to measure the entropy of an MNIST image. However most of the intensity values are 0, and <code>tf.log</code> returns <code>inf</code>. I know that I can simply calculate the entropy:</p>
  
  <pre><code>entropy = tf.reduce_mean(-tf.reduce_sum(X * tf.log(X), axis=1))
  </code></pre>
  
  <p>where X is a placeholder the MNIST image batch, defined as <code>X = tf.placeholder("float", [None, 784])</code>. However, the result is <code>nan</code> (many pixel values are 0, so <code>tf.log</code> will give <code>inf</code>). Is there a way to work around this, or another method to calculate the entropy of one image?</p>