title: Unable to load and use multiple keras models
body: <p>I'm trying to load three different models in the same process. Only the first one works as expected, the rest of them return like random results.
  Basically the order is as follows:</p>
  
  <ul>
  <li>define and compile first model</li>
  <li>load trained weights before</li>
  <li>rename layers</li>
  <li>the same process for the second model</li>
  <li>the same process for the third model</li>
  </ul>
  
  <p>So, something like:</p>
  
  <pre><code>model1 = Model(inputs=Input(shape=input_size_im) , outputs=layers_firstmodel)
  model1.compile(optimizer='sgd', loss='mse')
  model1.load_weights(weights_first, by_name=True)
  # rename layers but didn't work
  
  model2 = Model(inputs=Input(shape=input_size_im) , outputs=layers_secondmodel)
  model2.compile(optimizer='sgd', loss='mse')
  model2.load_weights(weights_second, by_name=True)
  # rename layers but didn't work
  
  model3 = Model(inputs=Input(shape=input_size_im) , outputs=layers_thirdmodel)
  model3.compile(optimizer='sgd', loss='mse')
  model3.load_weights(weights_third, by_name=True)
  # rename layers but didn't work
  
  for im in list_images:
      results_firstmodel = model1.predict(im) 
      results_secondmodel = model2.predict(im) 
      results_thirdmodel = model2.predict(im) 
  </code></pre>
  
  <p>I'd like to perform some inference over a bunch of images. To do that the idea consists in looping over the images and perform inference with these three algorithms, and return the results.</p>
  
  <p>I have tried to rename all layers to make them unique with no success. Also I  created a different graph for each network, and with a different session do the inference. This works but it's very inefficient (in addition I have to set their weights every time because of <code>sess.run(tf.global_variables_initializer())</code> removes them). Each time it's created a session tensorflow prints "creating tensorflow device (/device:GPU:0)".</p>
  
  <p>I am running Tensorflow 1.4.0-rc0, Keras 2.1.1 and Ubuntu 16.04 kernel 4.14.</p>