title: Writing your custom function for image preprocessing in Keras
body: <p>I am a beginner training an image dataset on diabetic retinopathy, using the <code>keras_flow_from_dataframe</code> class. But my model has been underfitting. So I tried preprocessing, by writing a custom preprocessing function to be passed in my image data generator class, using OpenCV's adaptive thresholding implementation. The function works very well when I use it outside of Keras, but when I add it to my image data generator class and fit my model, it returns a type error saying <code>bad argument type for built-in operation</code> before my first epoch starts.   </p>
  
  <p>Here's the preprocessing code:</p>
  
  <pre><code>def preprocess(im):
  
      im = cv2.imread(im, 1)
      im= cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
      im=cv2.resize(im, (300,300))
      im.resize(300, 300, 1)
      block_size = 73 
      constant = 2    
  # ADAPTIVE GAUSSIAN THRESHOLDING
  
      thr2 = cv2.adaptiveThreshold(im, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, block_size, constant)
      return thr2
  </code></pre>
  
  <p>It runs well outside of Keras when I test it with the images in my dataframe, but when I add it to my image data generator class, it throws an error.</p>
  
  <pre><code>train_datagen = ImageDataGenerator(
      rotation_range=30,
      width_shift_range=0.4,
      height_shift_range=0.4,
      shear_range=0.3,
      zoom_range=0.3,
      horizontal_flip = True,
      fill_mode='nearest',
      preprocessing_function = preprocess)
  
  valid_datagen = ImageDataGenerator(preprocessing_function = preprocess)
  </code></pre>
  
  <p>Then I load in my dataset from dataframe:</p>
  
  <pre><code>from keras.preprocessing.image import ImageDataGenerator
  
  traingen = train_datagen.flow_from_dataframe(x_train, x_col='path', y_col='level',class_mode='other', 
                                               target_size=(300,300), color_mode='grayscale', batch_size=16)
  
  validgen = valid_datagen.flow_from_dataframe(valid, x_col='path', y_col='level',class_mode='other',
                                              target_size=(300,300), color_mode='grayscale', batch_size=16)
  </code></pre>
  
  <p>Then I fit the model using <code>model.fit_generator</code>, which then throws me the type error: <code>bad argument type for built-in operation</code>.</p>
  
  <pre><code>TypeError                                 Traceback (most recent call last)
  &lt;ipython-input-126-30ceb84a2574&gt; in &lt;module&gt;()
        2 
        3 history = model.fit_generator(traingen, validation_data = validgen, epochs=100, steps_per_epoch=10,
  ----&gt; 4                                   validation_steps=10, verbose=1, callbacks=[lr_reduction])
        5 
        6 
  
  ~/var/python/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
       89                 warnings.warn('Update your `' + object_name + '` call to the ' +
       90                               'Keras 2 API: ' + signature, stacklevel=2)
  ---&gt; 91             return func(*args, **kwargs)
       92         wrapper._original_function = func
       93         return wrapper
  
  ~/var/python/lib/python3.6/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
     1416             use_multiprocessing=use_multiprocessing,
     1417             shuffle=shuffle,
  -&gt; 1418             initial_epoch=initial_epoch)
     1419 
     1420     @interfaces.legacy_generator_methods_support
  
  ~/var/python/lib/python3.6/site-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
      179             batch_index = 0
      180             while steps_done &lt; steps_per_epoch:
  --&gt; 181                 generator_output = next(output_generator)
      182 
      183                 if not hasattr(generator_output, '__len__'):
  
  ~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get(self)
      599         except Exception as e:
      600             self.stop()
  --&gt; 601             six.reraise(*sys.exc_info())
      602 
      603 
  
  ~/var/python/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
      691             if value.__traceback__ is not tb:
      692                 raise value.with_traceback(tb)
  --&gt; 693             raise value
      694         finally:
      695             value = None
  
  ~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get(self)
      593         try:
      594             while self.is_running():
  --&gt; 595                 inputs = self.queue.get(block=True).get()
      596                 self.queue.task_done()
      597                 if inputs is not None:
  
  ~/var/python/lib/python3.6/multiprocessing/pool.py in get(self, timeout)
      642             return self._value
      643         else:
  --&gt; 644             raise self._value
      645 
      646     def _set(self, i, obj):
  
  ~/var/python/lib/python3.6/multiprocessing/pool.py in worker(inqueue, outqueue, initializer, initargs, maxtasks, wrap_exception)
      117         job, i, func, args, kwds = task
      118         try:
  --&gt; 119             result = (True, func(*args, **kwds))
      120         except Exception as e:
      121             if wrap_exception and func is not _helper_reraises_exception:
  
  ~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get_index(uid, i)
      399         The value at index `i`.
      400     """
  --&gt; 401     return _SHARED_SEQUENCES[uid][i]
      402 
      403 
  
  ~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/iterator.py in __getitem__(self, idx)
       63         index_array = self.index_array[self.batch_size * idx:
       64                                        self.batch_size * (idx + 1)]
  ---&gt; 65         return self._get_batches_of_transformed_samples(index_array)
       66 
       67     def __len__(self):
  
  ~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/iterator.py in _get_batches_of_transformed_samples(self, index_array)
      233                 params = self.image_data_generator.get_random_transform(x.shape)
      234                 x = self.image_data_generator.apply_transform(x, params)
  --&gt; 235                 x = self.image_data_generator.standardize(x)
      236             batch_x[i] = x
      237         # optionally save augmented images to disk for debugging purposes
  
  ~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/image_data_generator.py in standardize(self, x)
      695         """
      696         if self.preprocessing_function:
  --&gt; 697             x = self.preprocessing_function(x)
      698         if self.rescale:
      699             x *= self.rescale
  
  &lt;ipython-input-112-7bddefa5e731&gt; in preprocess(im)
        1 def preprocess(im):
  ----&gt; 2     im = cv2.imread(im, 1)
        3     im= cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
        4     im=cv2.resize(im, (300,300))
        5     im.resize(300, 300, 1)
  
  TypeError: bad argument type for built-in operation
  
  
  TypeError                                 Traceback (most recent call last)
  &lt;ipython-input-126-30ceb84a2574&gt; in &lt;module&gt;()
        2 
        3 history = model.fit_generator(traingen, validation_data = validgen, epochs=100, steps_per_epoch=10,
  ----&gt; 4                                   validation_steps=10, verbose=1, callbacks=[lr_reduction])
        5 
        6 
  
  ~/var/python/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
       89                 warnings.warn('Update your `' + object_name + '` call to the ' +
       90                               'Keras 2 API: ' + signature, stacklevel=2)
  ---&gt; 91             return func(*args, **kwargs)
       92         wrapper._original_function = func
       93         return wrapper
  
  ~/var/python/lib/python3.6/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
     1416             use_multiprocessing=use_multiprocessing,
     1417             shuffle=shuffle,
  -&gt; 1418             initial_epoch=initial_epoch)
     1419 
     1420     @interfaces.legacy_generator_methods_support
  
  ~/var/python/lib/python3.6/site-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
      179             batch_index = 0
      180             while steps_done &lt; steps_per_epoch:
  --&gt; 181                 generator_output = next(output_generator)
      182 
      183                 if not hasattr(generator_output, '__len__'):
  
  ~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get(self)
      599         except Exception as e:
      600             self.stop()
  --&gt; 601             six.reraise(*sys.exc_info())
      602 
      603 
  
  ~/var/python/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
      691             if value.__traceback__ is not tb:
      692                 raise value.with_traceback(tb)
  --&gt; 693             raise value
      694         finally:
      695             value = None
  
  ~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get(self)
      593         try:
      594             while self.is_running():
  --&gt; 595                 inputs = self.queue.get(block=True).get()
      596                 self.queue.task_done()
      597                 if inputs is not None:
  
  ~/var/python/lib/python3.6/multiprocessing/pool.py in get(self, timeout)
      642             return self._value
      643         else:
  --&gt; 644             raise self._value
      645 
      646     def _set(self, i, obj):
  
  ~/var/python/lib/python3.6/multiprocessing/pool.py in worker(inqueue, outqueue, initializer, initargs, maxtasks, wrap_exception)
      117         job, i, func, args, kwds = task
      118         try:
  --&gt; 119             result = (True, func(*args, **kwds))
      120         except Exception as e:
      121             if wrap_exception and func is not _helper_reraises_exception:
  
  ~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get_index(uid, i)
      399         The value at index `i`.
      400     """
  --&gt; 401     return _SHARED_SEQUENCES[uid][i]
      402 
      403 
  
  ~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/iterator.py in __getitem__(self, idx)
       63         index_array = self.index_array[self.batch_size * idx:
       64                                        self.batch_size * (idx + 1)]
  ---&gt; 65         return self._get_batches_of_transformed_samples(index_array)
       66 
       67     def __len__(self):
  
  ~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/iterator.py in _get_batches_of_transformed_samples(self, index_array)
      233                 params = self.image_data_generator.get_random_transform(x.shape)
      234                 x = self.image_data_generator.apply_transform(x, params)
  --&gt; 235                 x = self.image_data_generator.standardize(x)
      236             batch_x[i] = x
      237         # optionally save augmented images to disk for debugging purposes
  
  ~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/image_data_generator.py in standardize(self, x)
      695         """
      696         if self.preprocessing_function:
  --&gt; 697             x = self.preprocessing_function(x)
      698         if self.rescale:
      699             x *= self.rescale
  
  &lt;ipython-input-112-7bddefa5e731&gt; in preprocess(im)
        1 def preprocess(im):
  ----&gt; 2     im = cv2.imread(im, 1)
        3     im= cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
        4     im=cv2.resize(im, (300,300))
        5     im.resize(300, 300, 1)
  
  TypeError: bad argument type for built-in operation
  
  
  TypeError                                 Traceback (most recent call last)
  &lt;ipython-input-126-30ceb84a2574&gt; in &lt;module&gt;()
        2 
        3 history = model.fit_generator(traingen, validation_data = validgen, epochs=100, steps_per_epoch=10,
  ----&gt; 4                                   validation_steps=10, verbose=1, callbacks=[lr_reduction])
        5 
        6 
  
  ~/var/python/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
       89                 warnings.warn('Update your `' + object_name + '` call to the ' +
       90                               'Keras 2 API: ' + signature, stacklevel=2)
  ---&gt; 91             return func(*args, **kwargs)
       92         wrapper._original_function = func
       93         return wrapper
  
  ~/var/python/lib/python3.6/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
     1416             use_multiprocessing=use_multiprocessing,
     1417             shuffle=shuffle,
  -&gt; 1418             initial_epoch=initial_epoch)
     1419 
     1420     @interfaces.legacy_generator_methods_support
  
  ~/var/python/lib/python3.6/site-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
      179             batch_index = 0
      180             while steps_done &lt; steps_per_epoch:
  --&gt; 181                 generator_output = next(output_generator)
      182 
      183                 if not hasattr(generator_output, '__len__'):
  
  ~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get(self)
      599         except Exception as e:
      600             self.stop()
  --&gt; 601             six.reraise(*sys.exc_info())
      602 
      603 
  
  ~/var/python/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
      691             if value.__traceback__ is not tb:
      692                 raise value.with_traceback(tb)
  --&gt; 693             raise value
      694         finally:
      695             value = None
  
  ~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get(self)
      593         try:
      594             while self.is_running():
  --&gt; 595                 inputs = self.queue.get(block=True).get()
      596                 self.queue.task_done()
      597                 if inputs is not None:
  
  ~/var/python/lib/python3.6/multiprocessing/pool.py in get(self, timeout)
      642             return self._value
      643         else:
  --&gt; 644             raise self._value
      645 
      646     def _set(self, i, obj):
  
  ~/var/python/lib/python3.6/multiprocessing/pool.py in worker(inqueue, outqueue, initializer, initargs, maxtasks, wrap_exception)
      117         job, i, func, args, kwds = task
      118         try:
  --&gt; 119             result = (True, func(*args, **kwds))
      120         except Exception as e:
      121             if wrap_exception and func is not _helper_reraises_exception:
  
  ~/var/python/lib/python3.6/site-packages/keras/utils/data_utils.py in get_index(uid, i)
      399         The value at index `i`.
      400     """
  --&gt; 401     return _SHARED_SEQUENCES[uid][i]
      402 
      403 
  
  ~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/iterator.py in __getitem__(self, idx)
       63         index_array = self.index_array[self.batch_size * idx:
       64                                        self.batch_size * (idx + 1)]
  ---&gt; 65         return self._get_batches_of_transformed_samples(index_array)
       66 
       67     def __len__(self):
  
  ~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/iterator.py in _get_batches_of_transformed_samples(self, index_array)
      233                 params = self.image_data_generator.get_random_transform(x.shape)
      234                 x = self.image_data_generator.apply_transform(x, params)
  --&gt; 235                 x = self.image_data_generator.standardize(x)
      236             batch_x[i] = x
      237         # optionally save augmented images to disk for debugging purposes
  
  ~/var/python/lib/python3.6/site-packages/keras_preprocessing/image/image_data_generator.py in standardize(self, x)
      695         """
      696         if self.preprocessing_function:
  --&gt; 697             x = self.preprocessing_function(x)
      698         if self.rescale:
      699             x *= self.rescale
  
  &lt;ipython-input-112-7bddefa5e731&gt; in preprocess(im)
        1 def preprocess(im):
  ----&gt; 2     im = cv2.imread(im, 1)
        3     im= cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
        4     im=cv2.resize(im, (300,300))
        5     im.resize(300, 300, 1)
  
  TypeError: bad argument type for built-in operation
  </code></pre>
  
  <p>I have also thought about preprocessing the images and saving them to a folder then I'll load them from that folder to a dataframe, but it's computationally expensive and time consuming.</p>