<p>They are basically the fullest learned model you can get from the network, before it's been squashed down to apply to only the number of classes we are interested in.  Check out how some researchers use them to train a shallow neural net based on what a deep network has learned:  <a href="https://arxiv.org/pdf/1312.6184.pdf" rel="noreferrer">https://arxiv.org/pdf/1312.6184.pdf</a></p>

<p>It's kind of like how when learning a subject in detail, you will learn a great many minor points, but then when teaching a student, you will try to compress it to the simplest case.  If the student now tried to teach, it'd be quite difficult, but would be able to describe it just well enough to use the language.</p>
What is the meaning of the word logits in TensorFlow?
<p>In the following TensorFlow function, we must feed the activation of artificial neurons in the final layer. That I understand. But I don't understand why it is called logits? Isn't that a mathematical function? </p>

<pre><code>loss_function = tf.nn.softmax_cross_entropy_with_logits(
     logits = last_layer,
     labels = target_output
)
</code></pre>