title: How to choose cross-entropy loss in TensorFlow?
body: <p>Classification problems, such as logistic regression or multinomial
  logistic regression, optimize a <strong>cross-entropy</strong> loss.
  Normally, the cross-entropy layer follows the <strong>softmax</strong> layer,
  which produces probability distribution.</p>
  
  <p>In tensorflow, there are at least a <strong>dozen of different cross-entropy loss functions</strong>:</p>
  
  <ul>
  <li><code>tf.losses.softmax_cross_entropy</code></li>
  <li><code>tf.losses.sparse_softmax_cross_entropy</code></li>
  <li><code>tf.losses.sigmoid_cross_entropy</code></li>
  <li><code>tf.contrib.losses.softmax_cross_entropy</code></li>
  <li><code>tf.contrib.losses.sigmoid_cross_entropy</code></li>
  <li><code>tf.nn.softmax_cross_entropy_with_logits</code></li>
  <li><code>tf.nn.sigmoid_cross_entropy_with_logits</code></li>
  <li>...</li>
  </ul>
  
  <p>Which one works only for binary classification and which are suitable for multi-class problems? When should you use <code>sigmoid</code> instead of <code>softmax</code>? How are <code>sparse</code> functions different from others and why is it only <code>softmax</code>?</p>
  
  <p>Related (more math-oriented) discussion: <a href="https://stackoverflow.com/q/44674847/712995">What are the differences between all these cross-entropy losses in Keras and TensorFlow?</a>. </p>