title: Using a custom R generator function with fit_generator (Keras, R)
body: <p>I'd like to train a convolutional network to solve a multi-class, multi-label problem on image data.  Due to the nature of the data, and for reasons I'll spare you, it would be best if I could use a custom R generator function to feed to the <code>fit_generator</code> command, instead of its built-in <code>image_data_generator</code> and <code>flow_images_from_directory</code> commands (which I was successfully able to get working, just not for this particular problem).</p>
  
  <p>Here (<a href="https://www.rdocumentation.org/packages/keras/versions/2.2.0/topics/fit_generator" rel="nofollow noreferrer">https://www.rdocumentation.org/packages/keras/versions/2.2.0/topics/fit_generator</a>) it says that I can do just that, without giving any examples.  So I tried the following.  Here is an extremely stripped down example of what I'm trying to do (this code is entirely self contained):</p>
  
  <pre><code>library(keras)
  library(reticulate)      #for py_iterator function
  
  play.network = keras_model_sequential() %&gt;%
    layer_dense(units = 10, activation = "relu", input_shape = c(10)) %&gt;%
    layer_dense(units = 1, activation = "relu")
  
  play.network %&gt;% compile(
    optimizer = "rmsprop",
    loss = "mse"
  )
  
  mikes.custom.generator.function = function()     #generates a 2-list of a random 1 x 10 array, and a scalar
  {
    new.func = function()
    {
      arr = array(dim = c(1,10))
      arr[,] = sample(1:10, 10, replace = TRUE)/10
      return(list(arr,runif(1)))
    }
  }
  
  mikes.custom.iterator = py_iterator(mikes.custom.generator.function())          #creates a python iterator object
  
  generator_next(mikes.custom.iterator)                 #correctly returns a 2-member list consisting of a 1 x 10 array, and a scalar
  generator_next(mikes.custom.iterator)[[1]]            #a 1 x 10 array
  generator_next(mikes.custom.iterator)[[2]]            #a scalar
  
  #try to fit with "fit_generator":
  
  play.network %&gt;% fit_generator(                       #FREEZES.
    mikes.custom.iterator,
    steps_per_epoch = 1,
    epochs = 1
  )
  </code></pre>
  
  <p>The thing freezes at training time, without giving me an error message or anything.  I also tried it with a custom image data generator for my original problem, same result.</p>
  
  <p>Note that this network trains just fine if I just use <code>fit</code> and input the training data manually:</p>
  
  <pre><code>play.network %&gt;% fit(generator_next(mikes.custom.iterator)[[1]],generator_next(mikes.custom.iterator)[[2]], epochs = 1, batch_size = 1)      
  
  #trains just fine
  </code></pre>
  
  <p>I think I know the problem, but I don't know the solution.  If you ask it for the class of my custom iterator, it gives</p>
  
  <pre><code>class(mikes.custom.iterator)
  
  [1] "python.builtin.iterator"  "rpytools.generator.RGenerator"  "python.builtin.object" 
  </code></pre>
  
  <p>whereas if I build an iterator using the builtin <code>image_data_generator</code> and <code>flow_images_from_directory</code> commands, it gives</p>
  
  <pre><code>train_datagen &lt;- image_data_generator(rescale = 1/255)
  
  class(train_datagen)
  
  [1] "keras.preprocessing.image.ImageDataGenerator" "keras_preprocessing.image.ImageDataGenerator" "python.builtin.object"
  
  train_generator &lt;- flow_images_from_directory(
    train_dir,
    train_datagen,
    ....
  )
  
  class(train_generator)
  
  [1] "python.builtin.iterator" "keras_preprocessing.image.DirectoryIterator" "keras_preprocessing.image.Iterator"        "tensorflow.python.keras.utils.data_utils.Sequence" "python.builtin.object"
  </code></pre>
  
  <p>So my guess is that <code>train_datagen</code> and/or <code>train_generator</code> have attributes that <code>mikes.custom.iterator</code> does not, and <code>fit_generator</code> is trying to call upon <code>mikes.custom.iterator</code> using functions other than the basic <code>generator_next</code> (which is in theory all it should really need).  But I don't know what they may be, or how to build <code>mikes.custom.iterator</code> correctly, even after searching for two hours online.</p>
  
  <p>Help anyone? </p>