title: Writing Custom Layer (3D RoI Pooling) using Keras in R
body: <p>Using R, I would like to produce a Keras custom layer that performs 3D Region of Interest (RoI) Pooling. My 3D RoI Pooling function works as expected outside of an R6Class KerasLayer, but I am having trouble integrating it into a Custom Layer. I am not sure if I am using autograph correctly in the below example. I am under the impressing that for me to use the R functionalities (i.e. for loop) in the "call" section of the custom layer I am required to wrap the function in autograph.</p>
  
  <p>Is the below custom layer for applying a 3D RoI Pooling (without training weights) appropriate syntex for an R user of Keras?</p>
  
  <p>The input for the 3d_RoI_Pool layer includes: </p>
  
  <p>(i) output layer from a VNet model, </p>
  
  <ul>
  <li>shape=(None, 16, 16, 40, 1)</li>
  </ul>
  
  <p>(ii) tensor defining the "index" for cropping the input layer to the extent of each RoI, </p>
  
  <pre><code>  - tf.Variable 'Variable:0' shape=(1, 1, 6) dtype=float64, numpy=array([[[ 5.,  5.,  4.,  5., 10., 20.]]])
  </code></pre>
  
  <p>(iii) output dimension for the pooled 3D RoIs. </p>
  
  <ul>
  <li>In this case its c(1, 1, 5, 5, 5, 1) for c(Batch_Size, RoI_Size, X, Y, Z, Channels)</li>
  </ul>
  
  <p>Below are some questions that I am unclear about:</p>
  
  <ul>
  <li><p>I initially apply layer_cropping_3d to crop each RoI but am not sure if custom layer is allowed to use available R Keras layers? For this reason, in code I performed my own cropping.</p></li>
  <li><p>Is it the correct use of autograph within custom layer?</p></li>
  <li><p>Should I instead use the layer_lambda() for such a function (i.e. Wraps arbitrary expression as a layer), given that I do not have trainable weights in the 3D_RoI_Pooling layer?</p></li>
  </ul>
  
  <p>Below is the Code that gets the output layer from a V-Net CNN and prepares inputs for the custom layer: </p>
  
  <pre><code>  # FINAL LAYERS OF V_Net
    Output_shortcut = up1_DeConv
    Final_Conv &lt;- up1_Concat %&gt;%
      # 1st
      layer_conv_3d(filters = Filter_Start, kernel_size = Kernel_Conv_Size, padding = "same", name = "Final_Conv") %&gt;%
      layer_batch_normalization() %&gt;%
      layer_activation("relu")
  
    Final_Conv &lt;- layer_add(c(Final_Conv, Output_shortcut), name = "Final_Conv_ResNet")
  
    # PREPARE INPUT FOR CUSTOM LAYER
    n_RoI &lt;- 1 
    Batch_Size &lt;- 1 
  
    Table_RoI &lt;- list(list(c(5, 5), c(4, 5), c(10, 20)))
    Table_RoI &lt;- array_reshape(unlist(Table_RoI), c(Batch_Size,n_RoI,6), order="F")
    Table_RoI &lt;- tf$Variable(Table_RoI, tf$int16)
  
    output_dim &lt;- c(Batch_Size, n_RoI, 5,5,5, as.numeric(dim(Final_Conv)[5]))
  
    # FEED CUSTOM LAYER TO CNN
    Final_Conv2 &lt;- Final_Conv %&gt;% layer_3D_ROI_pooled(Table_RoI=Table_RoI, output_dim= output_dim)
  </code></pre>
  
  <p>Below is the actual 3D RoI Pooling Custom Layer that generates an error:</p>
  
  <pre><code># 3D ROIpooled_Layer (R KERAS )
  ROIpooled_Layer &lt;- R6::R6Class("KerasLayer",
                                  inherit = KerasLayer,
                                  public = list(
                                    Table_RoI = NULL,
                                    output_dim = NULL, 
  
                                    ###############
                                    # INITIALISE
                                    ###############
  
                                    initialize = function(Table_RoI, output_dim) {
                                      self$Table_RoI = Table_RoI
                                      self$output_dim = output_dim
                                      },
  
                                    ###############
                                    # CALL FUNCTION   ROIpooled_Function &lt;-
                                    ###############
                                    call = autograph(function(x, mask = NULL,
                                                              Table_RoI,
                                                              output_dim) {
  
                                      ## Input_L ROI_Table 
                                      n_Batch &lt;- output_dim[1]
                                      n_RoI &lt;- output_dim[2]
                                      Channels &lt;- output_dim[6]
  
                                      for(r in 1:n_RoI){
  
  
                                          # layer_cropping_3d(Input_L, cropping = list(list(as.numeric(Table_RoI[,r,1]), as.numeric(Table_RoI[,r,2])), 
                                          #                                                               list(as.numeric(Table_RoI[,r,3]), as.numeric(Table_RoI[,r,4])), 
                                          #                                                               list(as.numeric(Table_RoI[,r,5]), as.numeric(Table_RoI[,r,6]))))
  
                                        RoI_Cropped &lt;-x[,(as.numeric(Table_RoI[,r,1])+1):(dim(feature_map_T)[2]-as.numeric(Table_RoI[,r,2])),
                                                                    (as.numeric(Table_RoI[,r,3])+1):(dim(feature_map_T)[3]-as.numeric(Table_RoI[,r,4])),
                                                                    (as.numeric(Table_RoI[,r,5])+1):(dim(feature_map_T)[4]-as.numeric(Table_RoI[,r,6])),]
  
                                        RoI_X_Res &lt;- as.array(k_shape(RoI_Cropped)[2])
                                        RoI_Y_Res &lt;- as.array(k_shape(RoI_Cropped)[3])
                                        RoI_Z_Res &lt;- as.array(k_shape(RoI_Cropped)[4])
  
                                        New_X_Res &lt;- as.array(output_dim[3])
                                        New_Y_Res &lt;- as.array(output_dim[4])
                                        New_Z_Res &lt;- as.array(output_dim[5])
  
                                        X_step = RoI_X_Res / New_X_Res    
                                        Y_step = RoI_Y_Res  / New_Y_Res  
                                        Z_step = RoI_Z_Res  / New_Z_Res  
  
                                        for(ch in 1:Channels) { 
                                          print(paste("ch", ch))
                                          for (k in 1:New_Z_Res) { 
                                            print(paste("k", k))
                                            for (j in 1:New_Y_Res) { 
                                              print(paste("j", j, "k", k))
                                              for (i in 1:New_X_Res) { 
                                                # INDEX X
                                                Index_Xstart &lt;- floor((i-1)*X_step+1)
                                                if(i+1 &lt;= RoI_X_Res){
                                                  Index_Xend &lt;- floor((i)*X_step)
                                                }else{
                                                  Index_Xend &lt;- RoI_X_Res
                                                }
                                                # INDEX Y
                                                Index_Ystart &lt;-  floor((j-1)*Y_step+1) 
                                                if(j+1 &lt;= RoI_Y_Res){
                                                  Index_Yend &lt;- floor((j)*Y_step)
                                                }else{
                                                  Index_Yend &lt;-RoI_Y_Res
                                                }
                                                # INDEX Z        
                                                Index_Zstart &lt;-  floor((k-1)*Z_step+1) 
                                                if(k+1 &lt;= RoI_Z_Res){
                                                  Index_Zend &lt;- floor((k)*Z_step)
                                                }else{
                                                  Index_Zend &lt;-RoI_Z_Res
                                                }
                                                Max_Pool_X_Value &lt;- as.array(k_max(RoI_Cropped[n_Batch,Index_Xstart:Index_Xend, Index_Ystart:Index_Yend, Index_Zstart:Index_Zend,ch])) # ADD BATCH AND CHANNEL LAYERS
                                                RoI_Pooled_Array[,r,i,j,k,ch] &lt;- Max_Pool_X_Value
                                              }# i LOOP
                                            } # j LOOP 
                                          } # k Loop
                                        } #Ch LOOP 
                                      } # r LOOP (ROI)
                                      feature_map_ROIpooled &lt;- tf$Variable(RoI_Pooled_Array, tf$int16) # ??? NOT SURE IF RETURN NEEDS TO BE A VARIABLE
  
                                      return (feature_map_ROIpooled)
  
                                    }), # END OF AUTO, # END OF CALL ... AUTOGRAPH FUNCTION
  
                                    ##############
                                    # OUTPUT SHAPE
                                    ##############
  
                                    compute_output_shape = function(input_shape) { 
                                      list(self$output_dim) 
                                      }
                                  )
                                )
  # 
  ###############################
  # Create layer wrapper function
  ###############################
  
  layer_3D_ROI_pooled &lt;- function(object, Table_RoI, output_dim, name = NULL, trainable = TRUE) {
    create_layer(ROIpooled_Layer, object, list(Table_RoI = Table_RoI, 
                                                output_dim = as.integer(output_dim), 
                                                name = name,
                                                trainable = FALSE
                                              ))
  }
  </code></pre>
  
  <p>ERROR:</p>
  
  <pre><code>Error in value[[3L]](cond) : 
     The R function's signature must not contains esoteric Python-incompatible constructs. Detailed traceback: SyntaxError: non-default argument follows default argument (&lt;string&gt;, line 3)
  </code></pre>
  
  <p>Any help/insight/clarity would be appreciated.</p>
  
  <p>Kind Regards,
  Dom</p>