title: Custom ImageDataGenerator keras
body: <p>I've been trying to implement Keras custom imagedatagenerator so that I can do hair and microscope image augmentation.</p>
  <p>This is the Datagenerator class:</p>
  <pre><code>class DataGenerator( Sequence ):
  
      def __init__(self,image_paths,labels, augmentations, batch_size=32, image_dimension=(224,224,3), shuffle=False):
          self.image_paths = image_paths
          self.labels = labels
          self.batch_size = batch_size
          self.image_dimension = image_dimension
          self.shuffle = shuffle
          self.augment = augmentations
  
      def __len__(self):
          return int(np.ceil(len(self.image_paths) / self.batch_size ))
  
      def _getitem__(self,index):
          indexes = self.indexes[index*self.batch_size : (index+1)*self.batch_size]
          batch_y = np.array([self.labels[k] for k in indexes])
          batch_x = [cv2.cvtColor(cv2.imread(self.image_paths[k]), cv2.COLOR_RGB2BGR) for k in indexes]
  
          return np.stack([
              self.augment(image=x)[&quot;image&quot;] for x in batch_x
          ], axis=0), np.array(batch_y)
  </code></pre>
  <p>Below Code is for albumentations augmentation (Just trying albualbumentations augmentation to test if the data generator works or not):</p>
  <pre><code>AUGMENTATIONS_TRAIN = Compose([
      HorizontalFlip(p=0.5),
      RandomContrast(limit=0.2, p=0.5),
      RandomGamma(gamma_limit=(80, 120), p=0.5),
      RandomBrightness(limit=0.2, p=0.5),
      HueSaturationValue(hue_shift_limit=5, sat_shift_limit=20,
                         val_shift_limit=10, p=.9),
      # CLAHE(p=1.0, clip_limit=2.0),
      ShiftScaleRotate(
          shift_limit=0.0625, scale_limit=0.1, 
          rotate_limit=15, border_mode=cv2.BORDER_REFLECT_101, p=0.8), 
      ToFloat(max_value=255)
  ])
  
      AUGMENTATIONS_TEST = Compose([
      # CLAHE(p=1.0, clip_limit=2.0),
      ToFloat(max_value=255)
  ])
  </code></pre>
  <p>Now creating DataGenerator object :</p>
  <pre><code>train_datagen = DataGenerator(  train['images'],
                                  train['target'],
                                  augmentations=AUGMENTATIONS_TRAIN,
                                  batch_size=32,
                                  image_dimension=(224,224,3) )
  val_datagen = DataGenerator(   validation['images'],
                                 validation['target'],
                                 augmentations=AUGMENTATIONS_TEST,
                                 batch_size=16,
                                 image_dimension=(224,224,3) )`
  
  </code></pre>
  <p>A NonImplementedError comes when i
  run <code>model.fit_generator(generator=train_datagen,steps_per_epoch=30,epochs = 30,validation_data=val_datagen,validation_steps=15) </code></p>
  <p>I have shared my kernel <a href="https://www.kaggle.com/prateekis1/keras-custom-augmentation" rel="nofollow noreferrer">here</a> and
  I was taking help from <a href="https://stanford.edu/%7Eshervine/blog/keras-how-to-generate-data-on-the-fly" rel="nofollow noreferrer">here</a>.
  I have also looked for other ways to augment which were all the same.</p>
  <p>I will be thankful if someone can tell why and where is the problem ? and Is there is any other good way to do custom image augmentation in keras.</p>