title: Tensorflow fails to initialize batch normalization layer
body: <p>I'm running Tensorflow 1.12 on GPU in a conda environment. I have several batch norm layers as part of convolution blocks defined in this way:</p>
  
  <pre><code>Conv=lambda NumFilter, Input, FilterSize=PARAMS['FilterSize'] : tf.layers.conv2d(Input, NumFilter, FilterSize, strides=1, activation=None, padding='SAME', use_bias=True, kernel_initializer=PARAMS['KernelInit'])
  
  def OneConv(layer,FilterNum,FilterSize,training):
      activate=tf.nn.relu(layer)
      norm=tf.layers.batch_normalization(activate,axis=-1,training=training)
      conv=Conv(FilterNum,norm,FilterSize)
      return conv
  
  def ConvBlock(BlockInput,name, FilterNum, training):
      with tf.name_scope(name):
  
          conv1=OneConv(BlockInput,FilterNum,PARAMS['FilterSize'],training)
          conc1=tf.concat([BlockInput,conv1],axis=-1)
  
          conv2=OneConv(conc1,FilterNum,PARAMS['FilterSize'],training)
          conc2=tf.concat([BlockInput,conv1,conv2],axis=-1)
  
          BlockOut=OneConv(conc2,FilterNum,1,training)
  
          return BlockOut
  </code></pre>
  
  <p>Which I use to build the Forward function. Whenever I try to test the network with:</p>
  
  <pre><code>X=tf.Variable(np.random.randn(1,128,128,1),dtype=tf.float32)
  
  init=tf.global_variables_initializer()
  test=Forward(X)
  
  with tf.Session() as sess:
      init.run()
      print(test.eval())
  </code></pre>
  
  <p>I get an error:</p>
  
  <blockquote>
    <p>FailedPreconditionError (see above for traceback): Attempting to use uninitialized value BatchNorm_12/beta
         [[node BatchNorm_12/beta/read (defined at /home/riccardo/.anaconda3/envs/Tenso/lib/python3.6/site-packages/tensorflow/contrib/framework/python/ops/variables.py:277)  = IdentityT=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]
         [[{{node ConvBlockUp1_1/conv2d_2/BiasAdd/_7}} = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_520_ConvBlockUp1_1/conv2d_2/BiasAdd", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]</p>
  </blockquote>
  
  <p>I can't quite figure out why the global initializer fails to initialize the batch norm layers, I also tried passing on specific initializers for the batchnorm parameters to tf.layers.batch_normalization but it did not change anything. Any idea what I am missing?</p>