title: Session graph empty - multiple graphs
body: <p>I am trying to use a Policy Network and CNN within the same Python file. I am only training the PN and using the CNN as a feed-forward feature extractor. I am getting an issue where the Train-PC isn't "building" the imported Keras graph into the session, and thus when I try to run a feed-forward function on the CNN (<code>feature_vector = self.extract_feature_vector([X_img_norm])[0]</code>), I get the error posted below. </p>
  
  <p>The only difference between the files on the Home-PC I am developing on and the Train-PC is the way I import the model which I believe may be causing this (possibly it's not recognised). I posted previously about this error <a href="https://stackoverflow.com/questions/53368717/loading-keras-model-keyerror-weighted-metrics">here</a>. </p>
  
  <p>Since the Train-PC is not my own, I cannot change the version of the packages. So I am looking for a solution which does not involve downgrading versions, as the issue with importing wasn't solved through that also.  </p>
  
  <p>Home-PC (Mac):</p>
  
  <pre><code>tensorflow==1.11.0
  keras 2.2.4
  </code></pre>
  
  <p>Train-PC (Ubuntu):</p>
  
  <pre><code>tensorflow-gpu==1.10.1
  keras 2.1.2
  </code></pre>
  
  <p>Code of CNN:</p>
  
  <pre><code>class ResNetCNN:
  
      def __init__(self):
          self.graph = tf.Graph()
          self.sess = tf.Session(graph=self.graph)
  
          with self.sess.graph.as_default():
              # Load pre-trained model from file
              self.CNN_ResNet_Pascal = models.load_model(CNN_MODEL_DIR)
              # self.CNN_ResNet_Pascal = tf.keras.models.load_model(CNN_MODEL_DIR)
  
              self.extract_feature_vector = K.function([self.CNN_ResNet_Pascal.layers[0].input],
                                                       [self.CNN_ResNet_Pascal.layers[-4].output])
  
      def feed_forward(self, img):
          """Extract the feature vector of the passed image using the pre-trained CNN."""
  
          # Resize observable region into input volume
          X_img = np.array(img, dtype=np.float32).reshape(-1, IMG_SIZE, IMG_SIZE, 3)
  
          # Normalise feature data
          X_img_norm = X_img / 255.0
  
          # Extract feature vector from the pre-trained CNN (1, 4096)
          feature_vector = self.extract_feature_vector([X_img_norm])[0]
  
          # Reshape tensor to (4096, )
          feature_vector = np.array(feature_vector).reshape(4096, )
  
          # Confirm that the correct output layer was used
          assert feature_vector.shape == (4096, ), "Incorrect CNN output layer: shape = {}".format(feature_vector.shape)
  
          return feature_vector
  </code></pre>
  
  <p>Error:</p>
  
  <pre><code>Traceback (most recent call last):
    File "keras_pn.py", line 856, in &lt;module&gt;
      s, a, r, d_r, n = rollout(epsilon, RENDER, PolicyNetwork, ResNetCNN)
    File "keras_pn.py", line 682, in rollout
      feature_vector = ResNetCNN.feed_forward(observable_region)
    File "keras_pn.py", line 87, in feed_forward
      feature_vector = self.extract_feature_vector([X_img_norm])[0]
    File "/home/name/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2357, in __call__
      **self.session_kwargs)
    File "/home/name/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 877, in run
      run_metadata_ptr)
    File "/home/name/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1025, in _run
      raise RuntimeError('The Session graph is empty.  Add operations to the '
  RuntimeError: The Session graph is empty.  Add operations to the graph before calling run().
  </code></pre>
  
  <p>Edit:</p>
  
  <pre><code>def __init__(self):
      self.graph = tf.Graph()
  
      with self.graph.as_default():
          # Load pre-trained model from file
          self.CNN_ResNet_Pascal = models.load_model(CNN_MODEL_DIR)
          # self.CNN_ResNet_Pascal = tf.keras.models.load_model(CNN_MODEL_DIR)
  
          self.extract_feature_vector = K.function([self.CNN_ResNet_Pascal.layers[0].input],
                                                   [self.CNN_ResNet_Pascal.layers[-4].output])
  
      self.sess = tf.Session(graph=self.graph)
  </code></pre>