title: Keras Image Preprocessing
body: <p>My training images are downscaled versions of their associated HR image. Thus, the input and the output images aren't the same dimension. For now, I'm using a hand-crafted sample of 13 images, but eventually I would like to be able to use my 500-ish HR (high-resolution) images dataset. This dataset, however, does not have images of the same dimension, so I'm guessing I'll have to crop them in order to obtain a uniform dimension.</p>
  
  <p>I currently have this code set up: it takes a bunch of <code>512x512x3</code> images and applies a few transformations to augment the data (flips). I thus obtain a basic set of 39 images in their HR form, and then I downscale them by a factor of 4, thus obtaining my trainset which consits of 39 images of dimension <code>128x128x3</code>.</p>
  
  <pre><code>import numpy as np
  
  from keras.preprocessing.image import ImageDataGenerator
  
  import matplotlib.image as mpimg
  import skimage
  from skimage import transform
  
  from constants import data_path
  from constants import img_width
  from constants import img_height
  
  from model import setUpModel
  
  
  def setUpImages():
  
      train = []
      finalTest = []
  
      sample_amnt = 11
      max_amnt = 13
  
      # Extracting images (512x512)
      for i in range(sample_amnt):
          train.append(mpimg.imread(data_path + str(i) + '.jpg'))
  
      for i in range(max_amnt-sample_amnt):
          finalTest.append(mpimg.imread(data_path + str(i+sample_amnt) + '.jpg'))
  
      # # TODO: https://keras.io/preprocessing/image/
      # ImageDataGenerator(featurewise_center=False, samplewise_center=False, featurewise_std_normalization=False,
      #                    samplewise_std_normalization=False, zca_whitening=False, zca_epsilon=1e-06, rotation_range=0,
      #                    width_shift_range=0.0, height_shift_range=0.0, brightness_range=None, shear_range=0.0,
      #                    zoom_range=0.0, channel_shift_range=0.0, fill_mode='nearest', cval=0.0, horizontal_flip=False,
      #                    vertical_flip=False, rescale=None, preprocessing_function=None, data_format=None,
      #                    validation_split=0.0, dtype=None)
  
      # Augmenting data
      trainData = dataAugmentation(train)
      testData  = dataAugmentation(finalTest)
  
      setUpData(trainData, testData)
  
  
  def setUpData(trainData, testData):
  
      # print(type(trainData))                          # &lt;class 'numpy.ndarray'&gt;
      # print(len(trainData))                           # 64
      # print(type(trainData[0]))                       # &lt;class 'numpy.ndarray'&gt;
      # print(trainData[0].shape)                       # (1400, 1400, 3)
      # print(trainData[len(trainData)//2-1].shape)     # (1400, 1400, 3)
      # print(trainData[len(trainData)//2].shape)       # (350, 350, 3)
      # print(trainData[len(trainData)-1].shape)        # (350, 350, 3)
  
      # TODO: substract mean of all images to all images
  
      # Separating the training data
      Y_train = trainData[:len(trainData)//2]    # First half is the unaltered data
      X_train = trainData[len(trainData)//2:]    # Second half is the deteriorated data
  
      # Separating the testing data
      Y_test = testData[:len(testData)//2]  # First half is the unaltered data
      X_test = testData[len(testData)//2:]  # Second half is the deteriorated data
  
      # Adjusting shapes for Keras input  # TODO: make into a function ?
      X_train = np.array([x for x in X_train])
      Y_train = np.array([x for x in Y_train])
      Y_test = np.array([x for x in Y_test])
      X_test = np.array([x for x in X_test])
  
      # # Sanity check: display four images (2x HR/LR)
      # plt.figure(figsize=(10, 10))
      # for i in range(2):
      #     plt.subplot(2, 2, i + 1)
      #     plt.imshow(Y_train[i], cmap=plt.cm.binary)
      # for i in range(2):
      #     plt.subplot(2, 2, i + 1 + 2)
      #     plt.imshow(X_train[i], cmap=plt.cm.binary)
      # plt.show()
  
      setUpModel(X_train, Y_train, X_test, Y_test)
  
  
  # TODO: possibly remove once Keras Preprocessing is integrated?
  def dataAugmentation(dataToAugment):
      print("Starting to augment data")
      arrayToFill = []
  
      # faster computation with values between 0 and 1 ?
      dataToAugment = np.divide(dataToAugment, 255.)
  
      # TODO: switch from RGB channels to CbCrY
      # # TODO: Try GrayScale
      # trainingData = np.array(
      #     [(cv2.cvtColor(np.uint8(x * 255), cv2.COLOR_BGR2GRAY) / 255).reshape(350, 350, 1) for x in trainingData])
      # validateData = np.array(
      #     [(cv2.cvtColor(np.uint8(x * 255), cv2.COLOR_BGR2GRAY) / 255).reshape(1400, 1400, 1) for x in validateData])
  
      # adding the normal images   (8)
      for i in range(len(dataToAugment)):
          arrayToFill.append(dataToAugment[i])
      # vertical axis flip         (-&gt; 16)
      for i in range(len(arrayToFill)):
          arrayToFill.append(np.fliplr(arrayToFill[i]))
      # horizontal axis flip       (-&gt; 32)
      for i in range(len(arrayToFill)):
          arrayToFill.append(np.flipud(arrayToFill[i]))
  
      # downsizing by scale of 4   (-&gt; 64 images of 128x128x3)
      for i in range(len(arrayToFill)):
          arrayToFill.append(skimage.transform.resize(
              arrayToFill[i],
              (img_width/4, img_height/4),
              mode='reflect',
              anti_aliasing=True))
  
      # # Sanity check: display the images
      # plt.figure(figsize=(10, 10))
      # for i in range(64):
      #     plt.subplot(8, 8, i + 1)
      #     plt.imshow(arrayToFill[i], cmap=plt.cm.binary)
      # plt.show()
  
      return np.array(arrayToFill)
  </code></pre>
  
  <p>My question is: <strong>in my case</strong>, can I use the Preprocessing tool that Keras offers? I would ideally like to be able to input my varying sized images of high quality, crop them (not downsize them) to <code>512x512x3</code>, and data augment them through flips and whatnot. Substracting the mean would also be part of what I'd like to achieve. That set would represent my validation set.</p>
  
  <p>Reusing the validation set, I want to downscale by a factor of 4 all the images, and that would generate my training set.</p>
  
  <p>Those two sets could then be split appropriately to obtain, ultimately, the famous <code>X_train</code> <code>Y_train</code> <code>X_test</code> <code>Y_test</code>.</p>
  
  <p>I'm just hesitant about throwing out all the work I've done so far to preprocess my mini sample, but I'm thinking if it can all be done with a single built-in function, maybe I should give that a go.</p>
  
  <p>This is my first ML project, hence me not understanding very well Keras, and the documentation isn't always the clearest. I'm thinking that the fact that I'm working with a X and Y that are different in size, maybe this function doesn't apply to my project.</p>
  
  <p>Thank you! :)</p>