<p>The keras documentation for <a href="https://keras.io/api/layers/normalization_layers/batch_normalization/" rel="nofollow noreferrer"><code>BatchNormalization</code></a> gives an answer to your question:</p>
<blockquote>
<p>Importantly, batch normalization works differently during training and
during inference.</p>
</blockquote>
<p>What happens during <em>training</em>, i.e. when calling <code>model.fit()</code>?</p>
<blockquote>
<p><strong>During training</strong> [...], the layer normalizes its output
using the mean and standard deviation of the current batch of inputs.</p>
</blockquote>
<p>But what will happen during inference, i.e. when calling <code>mode.predict()</code> as in your examples?</p>
<blockquote>
<p><strong>During inference</strong> [...], the layer normalizes its output using a moving average of
the mean and standard deviation of the batches it has seen during
training. That is to say, it returns <code>(batch - self.moving_mean) / (self.moving_var + epsilon) * gamma + beta</code>.</p>
<p><code>self.moving_mean</code> and <code>self.moving_var</code> are non-trainable variables that
are <strong>updated each time the layer in called in training mode</strong> [...].</p>
</blockquote>
<p>It's important to understand that batch normalization will calculate the statistics (mean and variance) of your whole training data during training by looking at statistics of single batches and internally updating the <code>moving_mean</code> and <code>moving_variance</code> parameters by a running average computed form the single batch statistics. Therefore they're not affected by backpropagation. Ideally, after your model has seen enough training examples (or did enough training epochs), <code>moving_mean</code> and <code>moving_variance</code> will correspond to the statistics of your whole training set. These two parameters are then used during inference to normalize test examples. At the start of training the two parameters will be initialized to 0 and 1. Further batch norm has two more parameters called gamma and beta, which will be updated by the optimizer and therefore depend on your loss.</p>
<p>In essence, <strong>yes</strong>, the output of batch normalization <em>during inference</em> is dependent on the number of epochs you have trained your model. Firstly, due to changing moving averages for mean and variance and second due to learned parameters gamma and beta.</p>
<p>For a deeper understanding of how batch normalization works and why it is needed, have a look at the <a href="https://arxiv.org/abs/1502.03167" rel="nofollow noreferrer">original publication</a>.</p>
Is output of Batch Normalization in Keras dependent on number of epochs?
<p>I am finding output of batchnormalization in Keras.
My model is:</p>
<p>#Import libraries</p>
<pre><code>import numpy as np
import keras
from keras import layers
from keras.layers import Input, Dense, Activation,  BatchNormalization, Flatten, Conv2D
from keras.models import Model
</code></pre>
<p>#Model</p>
<pre><code>def HappyModel3(input_shape):


    X_input = Input(input_shape, name='input_layer')
    X = BatchNormalization(axis = 1, name = 'batchnorm_layer')(X_input)
    
    X = Dense(1, activation='sigmoid', name='sigmoid_layer')(X)
    
    
    model = Model(inputs = X_input, outputs = X, name='HappyModel3')
    
    return model

    
</code></pre>
<p>Compiling Model | here number of epochs is 1</p>
<pre><code>X_train=np.array([[1,1,-1],[2,1,1]])
Y_train=np.array([0,1])

happyModel_1=HappyModel3(X_train[0].shape)
happyModel_1.compile(optimizer=keras.optimizers.RMSprop(), loss=keras.losses.mean_squared_error)
happyModel_1.fit(x = X_train, y = Y_train, epochs = 1 , batch_size = 2, verbose=0 )
</code></pre>
<p>finding Batch Normalisation layer's output for model with epochs=1:</p>
<pre><code>for i in range(0, len(happyModel_1.layers)):
    
    tmp_model = Model(happyModel_1.layers[0].input, happyModel_1.layers[i].output)
    tmp_output = tmp_model.predict(X_train)
    
    if i in (0,1) :
        print(happyModel_1.layers[i].name)
        print(tmp_output.shape)
        print(tmp_output)
        print('\n')
</code></pre>
<p>Code Output is:</p>
<pre><code>input_layer
(2, 3)
[[ 1.  1. -1.]
 [ 2.  1.  1.]]


batchnorm_layer
(2, 3)
[[ 0.99003249  0.99388224 -0.99551398]
 [ 1.99647105  0.99388224  0.9971655 ]]
</code></pre>
<p>We've normalized at axis=1 |
Batch Norm Layer Output: At axis=1, 1st dimension mean is 1.5, 2nd dimension mean is 1, 3rd dimension mean is 0.
Since its batch norm, I expect mean to be close to  0 for all 3 dimensions</p>
<p>This happens when I  increase epochs to 1000:</p>
<pre><code>happyModel_2=HappyModel3(X_train[0].shape)
happyModel_2.compile(optimizer=keras.optimizers.RMSprop(), loss=keras.losses.mean_squared_error)
happyModel_2.fit(x = X_train, y = Y_train, epochs = 1000 , batch_size = 2, verbose=0 )
</code></pre>
<p>finding Batch Normalisation layer's output for model with epochs=1000:</p>
<pre><code>for i in range(0, len(happyModel_2.layers)):
    tmp_model = Model(happyModel_2.layers[0].input, happyModel_2.layers[i].output)
    tmp_output = tmp_model.predict(X_train)
    
    if i in (0,1) :
        print(happyModel_2.layers[i].name)
        print(tmp_output.shape)
        print(tmp_output)
        print('\n')
</code></pre>
<p>#Code output</p>
<pre><code>input_layer
(2, 3)
[[ 1.  1. -1.]
 [ 2.  1.  1.]]


batchnorm_layer
(2, 3)
[[ -1.95576239e+00   8.08715820e-04  -1.86621261e+00]
 [  1.95795488e+00   8.08715820e-04   1.86590290e+00]]
</code></pre>
<p>We've normalized at axis=1 | Now At axis=1, batch norm layer output is: 1st dimension mean is 0, 2nd dimension mean is 0, 3rd dimension mean is 0. THIS IS AN EXPECTED OUTPUT NOW</p>
<p>My question is: Is output of Batch Normalization in Keras dependent on number of epochs?
(Probably YES, as we do backpropagation, batch Normalization parameters will be affected by increasing number of epochs)</p>