title: When to apply zero-centered data in an convolutional neural network
body: <p>I am building a cnn right now to classify grayscale images. The network did train until 97% accuracy on the test set until it got stuck. 
  Now i discovered that i forgot to zero-center the data before feeding it into the network. While thinking about where to best apply it i came up with more than one answer and like to have some thoughts on them. </p>
  
  <p>First a small overview over my architecture: 
  I am loading the datasets (train, validation, test) completely into the RAM. During runtime i load batches of images into placeholders which i then distort by randomly manipulating the brightness and contrast and then feed them into the network. </p>
  
  <p>Now, when i'd like to apply zero-centering, i could either take the mean of the training dataset and subtract it from every picture i feed into the network or i could subtract the mean of every single picture by using the tf.image.per_image_standardization() method from tensorflow. </p>
  
  <p>But if i calculate the mean of the training images, do i subtract it before or after i randomly distort the images? 
  And also if i use the network later to label real images, do i need to save the mean value and subtract it also from every picture i want to classify?</p>
  
  <p>Same goes for the per_image_standardization, do i apply it before or after distorting the images and do i also apply it on the real world data i want to classify later?</p>