title: Python, Keras custom loss function based on prediction, which is not the input
body: <p>I'm trying to write a custom loss function in keras, which is based on multiple predictions of the model during training.
  I have a previous list/dictionary like dict = {X1: [X2,X3,X4], X2:[X1, X6, X7]} etc.</p>
  
  <p>Given input X1, I need predictions of X2,X3,X4 during training.</p>
  
  <p>model.predict is not working, giving error
  ValueError: When feeding symbolic tensors to a model, we expect thetensors to have a static batch size. Got tensor with shape: (None, 36)</p>
  
  <pre class="lang-py prettyprint-override"><code>from keras.layers import * 
  from keras.models import Model
  import keras.backend as K
  
  input_tensor = Input(shape=(36,))
  hidden       = Dense(100, activation='relu')(input_tensor)
  out          = Dense(1, activation='linear')(hidden)
  
  def custom_loss(input_tensor, dict):
      def inner(y_true, y_pred):
          X2 = dict[input_tensor][0]
          X3 = dict[input_tensor][1]
          X4 = dict[input_tensor][2]
          X2_pred = model.predict(X2)
          X3_pred = model.predict(X3)
          X4_pred = model.predict(X4)
          return K.mean(max(X2_pred, X3_pred, X4_pred)-y_true)
  
      return inner
  
  custom_loss_final = custom_loss(input_tensor = input_tensor, dict = dict)
  
  model        = Model(input_tensor, out)
  model.compile(loss = custom_loss_final, optimizer='adam')
  
  model.fit(x = Train_X, y = Train_y, batch_size= 100)
  </code></pre>
  
  <p>Edit based on <strong>Anakin's</strong> solution:
  I tried your code, actually at np.append i needed to use axis = 0.</p>
  
  <p>now I have:
  INPUT_X.shape
   (100, 36)
  INPUT_Y.shape
  (100,3,36)</p>
  
  <p>INPUT_X: I have 100 training-samples, each a 36 len array.<br>
  INPUT_Y: Those are the X2, X3, etc.  </p>
  
  <p>actually I don't even need Y_true in my code, cause I will use the model(X2) etc.  </p>
  
  <p>I passed them to model.fit as you suggested, inside the loss function i printed some type/shapes out:<br>
   (input_tensor)<br>
  (?, 36)<br>
   ("y_true")<br>
  (?, ?)<br>
   (pred_y)<br>
  (?, 1)  </p>
  
  <p>I have no idea why y_true shape is (?,?) instead of (3,36). I can't put model(pred_y[:,0]), because I get:  </p>
  
  <p>ValueError: Shape must be rank 2 but is rank 1 for 'loss_56/dense_114_loss/model_57/dense_113/MatMul' (op: 'MatMul') with input shapes: [?], [36,300].<br>
   ##For clarity, 300 is the size of the hidden layer now.</p>