<p>Yes, <code>tf.layers.batch_normalization()</code> works with batches of single elements. Doing <em>batch normalization</em> on such batches is actually named <em>instance normalization</em> (i.e. normalization of a single instance).</p>

<p>@Maxim made a great <a href="https://stackoverflow.com/a/48118940/624547">post</a> about <em>instance normalization</em> if you want to know more. You can also find more theory on the web and in the literature, e.g. <a href="https://arxiv.org/abs/1607.08022" rel="nofollow noreferrer">Instance Normalization: The Missing Ingredient for Fast Stylization</a>.</p>
If the batch_size equals 1 in tf.layers.batch_normalization(), will it works correctly?
<p>everyone. I am using tensorflow 1.4 to train a model like <strong>U-net</strong> for my purpose. Due to the constraints of my hardware, when training, the <strong>batch_size could only set to be 1 otherwise there will be OOM error.</strong> </p>

<p>Here comes my question. In this case, when the batch_size equals to 1, will the <code>tf.layers.batch_normalization()</code> works correctly(saying moving average, moving variance, gamma, beta)? will small batch_size makes it working unstable?</p>

<p>In my work, I set <code>training=True</code> <strong>when training</strong>, and <code>training=False</code> <strong>when testing</strong>. When training, I use</p>

<pre><code>logits = mymodel.inference()
loss = tf.mean_square_error(labels, logits)
updata_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
with tf.control_dependencies(update_ops):
    train_op = optimizer.minimize(loss)
...

saver = tf.train.Saver(tf.global_variables())
with tf.Session() as sess:
    sess.run(tf.group(tf.global_variables_initializer(), 
                      tf.local_variables_initializer()))
    sess.run(train_op)
    ...
    saver.save(sess, save_path, global_step)
</code></pre>

<p>when testing, I use:</p>

<pre><code>logits = model.inference()
saver = tf.train.Saver()

with tf.Session() as sess:
    saver.restore(sess, checkpoint)
    sess.run(tf.local_variables_initializer())
    results = sess.run(logits)
</code></pre>

<p>Could anyone tell me that am I using this wrong? And how much influence with <strong>batch_size equals to 1</strong> in <strong>tf.layers.batch_normalization()</strong>?</p>

<p>Any help will be appreciated! Thanks in advance.</p>