title: Batch Normalization: Axis on which to take mean and variance
body: <p>I am trying to implement Batch Normalization (<a href="http://arxiv.org/pdf/1502.03167.pdf" rel="nofollow">http://arxiv.org/pdf/1502.03167.pdf</a>) in my convolutional neural network, but I am really confused as around what axis I should calculate mean and variance. </p>
  
  <p>If an input to the conv-layer is of shape 3 * 224 * 224 * 32<br/>
  where:<br/>
  3- input channels.<br/>
  224 * 224- shape of single channel<br/>
  32- minibatch size</p>
  
  <p><strong>What should be the axis in the following formula</strong><br/>
  Mean = numpy.mean(input_layer, axis= ? )</p>
  
  <p>And, if an input to the fully connected layer is of shape 100 * 32<br/>
  where:<br/>
  100- number of inputs<br/>
  32- minibatch size<br/></p>
  
  <p>Again, <strong>what should be the axis in the following formula</strong><br/>
  Mean = numpy.mean(input_layer, axis= ? )</p>