<p>I've increased learning rate to <code>0.01</code> and it seems like the network is able to learn something (I get <code>Epoch 1000/5000- 0s - loss: 0.2330</code>) .</p>

<p>I think it's worth to note the following from the abstract of original <a href="https://arxiv.org/pdf/1502.03167.pdf" rel="nofollow noreferrer">Batch Normalization</a> paper:</p>

<blockquote>
  <p>Batch Normalization allows us to use much higher learning rates and
  be less careful about initialization.  It also acts as a regularizer (...)</p>
</blockquote>

<p>That hinted to increased learning rate (that's something you might want to experiment with).</p>

<p>Be aware that since it works like regularization, BatchNorm <strong>should</strong> make your training loss worse - it's supposed to prevent overfitting and thus close the gap between the train and test/valid errors.</p>
Keras batch normalization stops convergence
<p>I'm new to keras and have been experimenting with various things such as BatchNormalization but it is not working at all. When the BatchNormalization line is commented out it will converge to around 0.04 loss or better, but with it as it is it will converge to 0.71 and get stuck around there, I'm not sure what's wrong.</p>

<pre><code>from sklearn import preprocessing
from sklearn.datasets import load_boston
from keras.models import Model
from keras.layers import Input, Dense
from keras.layers.normalization import BatchNormalization
import keras.optimizers

boston = load_boston()

x = boston.data
y = boston.target

normx = preprocessing.scale(x)
normy = preprocessing.scale(y)

# doesnt construct output layer
def layer_looper(inputs, number_of_loops, neurons):
    inputs_copy = inputs
    for i in range(number_of_loops):

        inputs_copy = Dense(neurons, activation='relu')(inputs_copy)
        inputs_copy = BatchNormalization()(inputs_copy)

    return inputs_copy

inputs = Input(shape = (13,))

x = layer_looper(inputs, 40, 20)

predictions = Dense(1, activation='linear')(x)

model = Model(inputs=inputs, outputs=predictions)

opti = keras.optimizers.Adam(lr=0.0001)
model.compile(loss='mean_absolute_error', optimizer=opti, metrics=['acc'])

print(model.summary())

model.fit(normx, normy, epochs=5000, verbose=2, batch_size=128)
</code></pre>

<p>I have tried experimenting with batch sizes and the optimizer but it doesn't seem very effective. Am I doing something wrong?</p>