title: Training built-in convnets on array data in R keras
body: <p>I am trying to train a built-in convnet architecture on my own data in R keras. My data is stored in an array in R, rather than in individual image files, which seems to be the standard.</p>
  <p>I think my main problem is that I don't know how to preprocess my feature data correctly.</p>
  <p>Here is an simple example for data and model definition (which works):</p>
  <pre><code>#simulate data resembling images, but in array format:
  p &lt;- 32 # note: minium height/width for resnet
  toy_x &lt;- array(runif(p*p*100*3), c(100, p, p, 3))
  toy_y &lt;- runif(100)
  
  #define and compile model
  input &lt;- layer_input(shape = c(p, p, 3))
  N1 &lt;- application_resnet50(weights = NULL,
                                 input_tensor = input,
                                 include_top = FALSE)
  output_layer_instance &lt;- layer_dense(units = 1, activation = 'sigmoid')
  output &lt;- input %&gt;% N1() %&gt;% output_layer_instance()
  model &lt;- keras_model(input, output)
  model %&gt;% compile(loss = &quot;binary_crossentropy&quot;, optimizer = &quot;adam&quot;)
  </code></pre>
  <p>But when I try to fit the model using the following code, I get an error:</p>
  <pre><code>model %&gt;% fit(toy_x, toy_y, epochs = 1)
  </code></pre>
  <p>I'm not sure the error is very informative, but here it is:</p>
  <pre><code> Error in py_call_impl(callable, dots$args, dots$keywords) : 
    ValueError: in user code:
  
      /root/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py:571 train_function  *
          outputs = self.distribute_strategy.run(
      /root/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages/tensorflow/python/distribute/distribute_lib.py:951 run  **
          return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
      /root/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages/tensorflow/python/distribute/distribute_lib.py:2290 call_for_each_replica
          return self._call_for_each_replica(fn, args, kwargs)
      /root/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages/tensorflow/python/distribute/distribute_lib.py:2649 _call_for_each_replica
          return fn(*args, **kwargs)
      /root/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py:533 train_step 
  </code></pre>
  <p>I have tried a few alternative solutions. As mentioned above, I think the issue may be due to lack of preprocessing of the feature data. I have tried using the built-in preprocessing function, but without luck - I get the same error as above from running the following:</p>
  <pre><code>toy_x_preproc &lt;- imagenet_preprocess_input(toy_x)
  model %&gt;% fit(toy_x_preproc, toy_y, epochs = 1)
  </code></pre>
  <p>I have also tested that the code runs without using the built-in example resnet by replacing it with a simple convnet (still using the functional API):</p>
  <pre><code>#define &amp; compile model
  model2_input &lt;- layer_input(shape = c(p, p, 3))
  model2_output &lt;- model2_input %&gt;% 
    layer_conv_2d(filters = 25, kernel_size = c(2,2), activation = &quot;relu&quot;, 
                             input_shape = c(p,p,1)) %&gt;% 
    layer_max_pooling_2d(pool_size = c(2, 2)) %&gt;% 
    layer_flatten() %&gt;% 
    layer_dense(units = 1, activation = 'sigmoid')  
  model2 &lt;- keras_model(model2_input, model2_output)
  model2 %&gt;% compile(
    loss = &quot;binary_crossentropy&quot;,
    optimizer = &quot;adam&quot;)
  
  #train on &quot;raw&quot; toy_x -- works
  model2 %&gt;% fit(toy_x, toy_y, epochs = 1)
  </code></pre>
  <p>This runs without an error. It also works if I rerun the entire chunk but fit on <code>toy_x_preproc</code> instead.</p>
  <p>Thank you for reading - and I will greatly appreciate any help.</p>