title: PyTorch LogSoftmax vs Softmax for CrossEntropyLoss
body: <p>I understand that PyTorch's LogSoftmax function is basically just a more numerically stable way to compute <code>Log(Softmax(x))</code>. Softmax lets you convert the output from a Linear layer into a categorical probability distribution.</p>
  <p>The <a href="https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html" rel="noreferrer">pytorch documentation</a> says that CrossEntropyLoss combines <code>nn.LogSoftmax()</code> and <code>nn.NLLLoss()</code> in one single class.</p>
  <p>Looking at <code>NLLLoss</code>, I'm still confused...Are there 2 logs being used? I think of negative log as information content of an event. (As in <a href="https://en.wikipedia.org/wiki/Entropy_(information_theory)" rel="noreferrer">entropy</a>)</p>
  <p>After a bit more looking, I think that <code>NLLLoss</code> assumes that you're actually passing in log probabilities instead of just probabilities. Is this correct? It's kind of weird if so...</p>