title: Load custom loss with extra input in keras
body: <p>I have a custom loss function that takes the input to the model as one of the arguments. If I load in the same session in which I train, I can load it no problem using <a href="https://stackoverflow.com/questions/48373845/loading-model-with-custom-loss-keras">this</a> technique.</p>
  
  <pre class="lang-py prettyprint-override"><code>
  def custom_loss(inputs):
      def loss(y_true, y_pred):
          return ...
      return loss
  
  inputs = keras.layers.Input(shape=(...))
  y = keras.layers.Activation('tanh')(inputs)
  
  model = keras.models.Model(inputs=inputs, outputs=y)
  
  model.compile(loss=custom_loss(inputs), optimizer='Adam')
  model.fit(...)
  model.save('mymodel.h5')
  load_model('mymodel.h5', custom_objects={'custom_loss': custom_loss(inputs})
  </code></pre>
  
  <p>However, I run into problems when I try to load the model in a later session, because this time I don't have access to the original input tensor. If I make a new inputs placeholder, then the model expects two different sets of inputs and I error out.</p>
  
  <pre><code>inputs = keras.layers.Input(shape=(...))
  load_model('mymodel.h5', custom_objects={'custom_loss': custom_loss(inputs)})
  </code></pre>
  
  <p>Is there a good way to solve this problem? The issue at the end of the day is that the inputs haven't been deserialized yet so they can't be passed in to the custom objects.</p>
  
  <p>I don't want to just save the weights and create a new model with the same weights because I lose the optimizer state.</p>