title: Cannot concatenate Keras Lambda layers
body: <p>I need to process some layers in a different way, doing some OR operations. I've found how to do it, I create a Lambda Layer and process the data with <code>keras.backend.any</code>. I am also doing a split, because I need to operate 2 separates groups with my logical OR.</p>
  
  <pre><code>def logical_or_layer(x):
      """Processing an OR operation"""
      import keras.backend
      #normalized to 0,1 
      aux_array = keras.backend.sign(x)
      aux_array = keras.backend.relu(aux_array)
      # OR operation
      aux_array = keras.backend.any(aux_array)
      # casting back the True/False to 1,0
      aux_array = keras.backend.cast(aux_array, dtype='float32')
  
      return aux_array
  </code></pre>
  
  <p>Then I'am creating my layers like this:</p>
  
  <pre><code>#this is the input tensor
  inputs = Input(shape=(inputSize,))
  
  #this is the Neurule layer
  x = Dense(neurulesQt, activation='softsign')(inputs)
  #after each neurule layer, the outputs need to be put into SIGNUM (-1 or 1)
  x = Lambda(signumTransform, output_shape=lambda x:x, name='signumAfterNeurules')(x)
  
  #separating into 2 (2 possible outputs)
  layer_split0 = Lambda( lambda x: x[:, :end_output0], output_shape=(11, ), name='layer_split0')(x)
  layer_split1 = Lambda( lambda x: x[:, start_output1:end_output1], output_shape=(9,), name='layer_split1')(x)
  
  #this is the OR layer
  y_0 = Lambda(logical_or_layer, output_shape=(1,), name='or0')(layer_split0)
  y_1 = Lambda(logical_or_layer, output_shape=(1,), name='or1')(layer_split1)
  </code></pre>
  
  <p>Just FYI: Neurules are Neurons created based on IF-THEN rules, that's one project to work with Neurons which were trained with a TruthTable, representing Expert Knowledge.</p>
  
  <p>Now, when I try to put the splitted layers back like this:</p>
  
  <pre><code>y = concatenate([y_0,y_1])
  </code></pre>
  
  <p>This error comes:</p>
  
  <pre><code>ValueError: Can't concatenate scalars (use tf.stack instead) for 'concatenate_32/concat' (op: 'ConcatV2') with input shapes: [], [], [].
  </code></pre>
  
  <p>Then ok, let's use the <code>tf.stack</code> as suggested:</p>
  
  <pre><code>y = keras.backend.stack([y_0, y_1])
  </code></pre>
  
  <p>Then it can't be used as an output in the Model anymore, when I try:</p>
  
  <pre><code>model = Model(inputs=inputs, outputs=y)
  </code></pre>
  
  <p>Comes the error:</p>
  
  <pre><code>ValueError: Output tensors to a Model must be the output of a Keras `Layer` (thus holding past layer metadata). Found: Tensor("stack_14:0", shape=(2,), dtype=float32)
  </code></pre>
  
  <p>Checking with the function <code>keras.backend.is_keras_tensor(y)</code> it gives me <code>False</code>, but with all the other layers it gives me <code>True</code></p>
  
  <p>How should I concatenate it correctly?</p>
  
  <p><strong>EDIT:</strong> Following the answer from @today I was able to create a new Lambda Layer with the <code>stack</code> wrapped inside of it. But the outputs are modified, it should be <code>(None,2)</code> and it is <code>(2,None,1)</code> here is the output from <code>model.summary()</code>:</p>
  
  <pre><code>__________________________________________________________________________________________________
  Layer (type)                    Output Shape         Param #     Connected to                     
  ==================================================================================================
  input_90 (InputLayer)           (None, 24)           0                                            
  __________________________________________________________________________________________________
  dense_90 (Dense)                (None, 20)           500         input_90[0][0]                   
  __________________________________________________________________________________________________
  signumAfterNeurules (Lambda)    (None, 20)           0           dense_90[0][0]                   
  __________________________________________________________________________________________________
  layer_split0 (Lambda)           (None, 11)           0           signumAfterNeurules[0][0]        
  __________________________________________________________________________________________________
  layer_split1 (Lambda)           (None, 9)            0           signumAfterNeurules[0][0]        
  __________________________________________________________________________________________________
  or0 (Lambda)                    (None, 1)            0           layer_split0[0][0]               
  __________________________________________________________________________________________________
  or1 (Lambda)                    (None, 1)            0           layer_split1[0][0]               
  __________________________________________________________________________________________________
  output (Lambda)                 (2, None, 1)         0           or0[0][0]                        
                                                                   or1[0][0]                        
  ==================================================================================================
  Total params: 500
  Trainable params: 0
  Non-trainable params: 500
  __________________________________________________________________________________________________
  </code></pre>
  
  <p>How should I define the output_shape in the layers to have the batch still there at the end?</p>
  
  <p><strong>EDIT2:</strong> Following the tips from @today I've done the following:</p>
  
  <pre><code>#this is the input tensor
  inputs = Input(shape=(inputSize,))
  
  #this is the Neurule layer
  x = Dense(neurulesQt, activation='softsign')(inputs)
  #after each neuron layer, the outputs need to be put into SIGNUM (-1 or 1)
  x = Lambda(signumTransform, output_shape=lambda x:x, name='signumAfterNeurules')(x)
  #separating into 2 (2 possible outputs)
  layer_split0 = Lambda( lambda x: x[:, :end_output0], output_shape=[11], name='layer_split0')(x)    
  layer_split1 = Lambda( lambda x: x[:, start_output1:end_output1], output_shape=[9], name='layer_split1')(x)
  #this is the OR layer
  y_0 = Lambda(logical_or_layer, output_shape=(1,), name='or0')(layer_split0)
  y_1 = Lambda(logical_or_layer, output_shape=(1,), name='or1')(layer_split1)
  
  y = Lambda(lambda x: K.stack([x[0], x[1]]),output_shape=(2,),  name="output")([y_0, y_1])
  </code></pre>
  
  <p>Now it seems to work correctly, the <code>model.summary()</code> below:</p>
  
  <pre><code>__________________________________________________________________________________________________
  Layer (type)                    Output Shape         Param #     Connected to                     
  ==================================================================================================
  input_1 (InputLayer)            (None, 24)           0                                            
  __________________________________________________________________________________________________
  dense_1 (Dense)                 (None, 20)           500         input_1[0][0]                    
  __________________________________________________________________________________________________
  signumAfterNeurules (Lambda)    (None, 20)           0           dense_1[0][0]                    
  __________________________________________________________________________________________________
  layer_split0 (Lambda)           (None, 11)           0           signumAfterNeurules[0][0]        
  __________________________________________________________________________________________________
  layer_split1 (Lambda)           (None, 9)            0           signumAfterNeurules[0][0]        
  __________________________________________________________________________________________________
  or0 (Lambda)                    (None, 1)            0           layer_split0[0][0]               
  __________________________________________________________________________________________________
  or1 (Lambda)                    (None, 1)            0           layer_split1[0][0]               
  __________________________________________________________________________________________________
  output (Lambda)                 (None, 2)            0           or0[0][0]                        
                                                                   or1[0][0]                        
  ==================================================================================================
  Total params: 500
  Trainable params: 0
  Non-trainable params: 500
  __________________________________________________________________________________________________
  </code></pre>