<p>In <a href="https://arxiv.org/pdf/2105.07576.pdf" rel="nofollow noreferrer">Rethinking “Batch” in BatchNorm</a> research carried out by FAIR, batch normalization and batch size are discussed. According to the graph  below, you can see the relation of batch normalization and batch size. It shows that when you use smaller batch size you do not need to use batch normalization. Batch normalization is helpful when you have bigger batch size. Using smaller batch size with batch normalization leads to training/testing inconsistency.</p>
<p><a href="https://i.stack.imgur.com/aP50D.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/aP50D.png" alt="enter image description here" /></a></p>
<p>Classification error under different normalization batch
sizes, with a fixed total batch size of 1024. <strong>Green</strong>: error rate on
unaugmented training set using mini-batch statistics; <strong>Red</strong>: error
rate on validation set using population statistics estimated by PreciseBN; <strong>Blue</strong>: <strong>error rate on validation set</strong> using mini-batch statistics of random batches (with the same normalization batch size
used in training). The gap between red and blue curves is caused
by <em><strong>train-test inconsistency</strong></em>, while the gap between blue and green
curves is the generalization gap on unseen dataset.</p>
batch size without batch normalization
<p>I'm working on image super-resolution tasks with EDSR as a baseline model. Following EDSR, I'm not using any batch-norm layers in my model. I suddenly came up with a stupid question about batch-sizes.</p>
<p>Currently, I'm training my model with batch-size=32 (as in EDSR). But since I'm not using any batch-normalization technique, I cant see any reason for using batch sizes greater than 1. But I'm not confident with my thoughts since the author's implementations <strong>are</strong> using batch sizes greater than 1.</p>
<p>Could someone help me with this? What am I missing?</p>