<p>Having <code>100%</code> accuracy on train dataset while having <code>80%</code> accuracy on test dataset doesn't mean that your model overfits. Moreover, it almost surely doesn't overfit if your model is equipped with much more effective parameters that the number of training samples <code>[2]</code>, <code>[5]</code> (insanely large model example <code>[1]</code>). This contradicts to conventional statistical learning theory, but these are empirical results.</p>

<p>For models with number of parameters greater than number of samples, it's better to continue to optimize the logistic or cross-entropy loss even after the training error is zero and the training loss is extremely small, and even if the validation loss increases <code>[3]</code>. This may hold even regardless of batch size <code>[4]</code>.</p>

<h3>Clarifications (edit)</h3>

<ul>
<li>The "models" I was referring to are neural networks with two or more hidden layers (could be also convolutional layers prior to dense layers).</li>
<li><code>[1]</code> is cited to show a clear contradiction to classical statistical learning theory, which says that large models may overfit without some form of regularization.</li>
<li>I would invite anyone who disagrees with <em>"almost surely doesn't overfit"</em> to provide a reproducible example where models, say for MNIST/CIFAR etc with few hundred thousand parameters do overfit (in a sense of increasing with iterations test error curve).</li>
</ul>

<p><code>[1]</code> <a href="https://arxiv.org/pdf/1701.06538.pdf" rel="nofollow noreferrer">Noam Shazeer,  Azalia Mirhoseini,  Krzysztof Maziarz,  Andy Davis,  Quoc V. Le,Geoffrey  E.  Hinton,  and  Jeff  Dean.   Outrageously  large  neural  networks:   Thesparsely-gated mixture-of-experts layer.CoRR, abs/1701.06538, 2017.</a></p>

<p><code>[2]</code> <a href="https://www.padl.ws/papers/Paper%2010.pdf" rel="nofollow noreferrer">Lei Wu, Zhanxing Zhu, et al. Towards understanding generalization of deep learn-ing:  Perspective of loss landscapes.arXiv preprint arXiv:1706.10239, 2017.</a></p>

<p><code>[3]</code> <a href="https://arxiv.org/pdf/1710.10345.pdf" rel="nofollow noreferrer"> Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasekar, and NathanSrebro.  The implicit bias of gradient descent on separable data.The Journal of Machine Learning Research, 19(1):2822–2878, 2018.</a></p>

<p><code>[4]</code> <a href="https://arxiv.org/pdf/1705.08741.pdf" rel="nofollow noreferrer"> Elad Hoffer, Itay Hubara, and Daniel Soudry. Train longer, generalize better: clos-ing the generalization gap in large batch training of neural networks.  InAdvancesin Neural Information Processing Systems, pages 1731–1741, 2017.</a>`</p>

<p><code>[5]</code> <a href="https://arxiv.org/pdf/1611.03530.pdf" rel="nofollow noreferrer">Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals.Understanding  deep  learning  requires  rethinking  generalization.arXiv preprintarXiv:1611.03530, 2016.</a></p>
What am I trying to do here? train acc: 100%, test acc: 80% does this mean overfitting?
<pre><code>classifier.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
classifier.fit(X_train, y_train, epochs=50, batch_size=100)

Epoch 1/50
27455/27455 [==============================] - 3s 101us/step - loss: 2.9622 - acc: 0.5374
</code></pre>

<p>I know I'm compiling my model in first line and fitting it in second. I know what is optimiser. I'm interested the meaning of <code>metrics=['accuracy']</code> and what does the <code>acc: XXX</code> exactly mean when I compile the model. 
Also, I'm getting <code>acc : 1.000</code> when I train my model (100%) but when I test my model I'm getting 80% accuracy. Does my model overfitting? </p>