<p>The idea is that it is harder to overfit due to <em>gradient noise</em>. But it is not only improving. See Table 5.9 on page 59 of <a href="https://arxiv.org/pdf/1707.09725.pdf" rel="nofollow noreferrer">Analysis and Optimization of Convolutional Neural Network Architectures</a>. If you make too small batch sizes, the accuracy decreases again.</p>
Deep Learning: Why does the accuracy gets better as batch size decreases?
<p>My understanding about batch size was the the smaller, the noisier and the less computationally efficient, however I developed a model and I'm using a certain dataset in which I try different configurations, and all I can see is that the accuracy gets better as the batch size decreases (while keeping the rest of the parameters constant). I tried batch sizes of 2, 4, 8, 16, 32 and 64. I expected that the accuracy would increase from 2-8, and it would be stable/oscillating in the others, but the improvement over the reduction of the batch size is totally clear (2 times 5-fold cross-validation).</p>

<p>My question is, why is this happening? What can I say about my model and dataset when this is happening?</p>