<p>I want to add two points:</p>

<p>1) When use special treatments, it is possible to achieve similar performance for a very large batch size while speeding-up the training process tremendously. For example, 
<a href="https://arxiv.org/pdf/1706.02677.pdf" rel="nofollow noreferrer"><em>Accurate, Large Minibatch SGD:Training ImageNet in 1 Hour</em></a></p>

<p>2) Regarding your MNIST example, I really don't suggest you to over-read these numbers. Because the difference is so subtle that it could be caused by noise. I bet if you try models saved on a different epoch, you will see a different result. </p>
Why does different batch-sizes give different accuracy in Keras?
<p>I was using Keras' CNN to classify MNIST dataset. I found that using different batch-sizes gave different accuracies. Why is it so?  </p>

<p><a href="https://i.stack.imgur.com/Ad4oO.png" rel="nofollow noreferrer">Using Batch-size 1000</a> (Acc = 0.97600)</p>

<p><a href="https://i.stack.imgur.com/CRLd3.png" rel="nofollow noreferrer">Using Batch-size 10</a>  (Acc = 0.97599)</p>

<p>Although, the difference is very small, why is there even a difference?
<strong>EDIT - I have found that the difference is only because of precision issues and they are in fact equal.</strong></p>