<p>If your target vocabulary(or in other words amount of classes you want to predict) is really big, it is very hard to use regular softmax, because you have to calculate probability for every word in dictionary. By Using  <code>sampled_softmax_loss</code> you only take in account subset <strong>V</strong> of your vocabulary to calculate your loss.</p>

<p>Sampled softmax only makes sense if we sample(our <strong>V</strong>) less than vocabulary size. If your vocabulary(amount of labels) is small, there is no point using <code>sampled_softmax_loss</code>.</p>

<p>You can see implementation details in this paper:
<a href="http://arxiv.org/pdf/1412.2007v2.pdf" rel="noreferrer">http://arxiv.org/pdf/1412.2007v2.pdf</a></p>

<p>Also you can see example where it is used - Sequence to sequence translation in this <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/models/rnn/translate/seq2seq_model.py" rel="noreferrer">example</a></p>
In Tensorflow, what is the difference between sampled_softmax_loss and softmax_cross_entropy_with_logits
<p>In tensorflow, there are methods called <a href="https://www.tensorflow.org/versions/master/api_docs/python/tf/nn/softmax_cross_entropy_with_logits" rel="nofollow noreferrer"><code>softmax_cross_entropy_with_logits</code></a> and <a href="https://www.tensorflow.org/versions/master/api_docs/python/tf/nn/sampled_softmax_loss" rel="nofollow noreferrer"><code>sampled_softmax_loss</code></a>.</p>

<p>I read the tensorflow document and searched google for more information but I couldn't find the difference. It looks like to me both calculates the loss using softmax function.</p>

<h3>Using <code>sampled_softmax_loss</code> to calculate the loss</h3>

<pre><code>loss = tf.reduce_mean(tf.nn.sampled_softmax_loss(...))
</code></pre>

<h3>Using <code>softmax_cross_entropy_with_logits</code> to calculate the loss</h3>

<pre><code>loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(P, Q))
</code></pre>

<p>To me, calculating softmax loss is same as calculating softmaxed cross entropy (e.g. <code>cross_entropy(softmax(train_x))</code>)</p>

<p>Could somebody tell me the why there is two different methods and which method should I use in which case?</p>