title: TensorFlow: Restoring variables from from multiple checkpoints
body: <p>I have the following situation:</p>
  
  <ul>
  <li><p>I have 2 models written in 2 separate scripts:</p></li>
  <li><p>Model A consists of variables <code>a1</code>, <code>a2</code>, and <code>a3</code>, and is written in <code>A.py</code></p></li>
  <li><p>Model B consists of variables <code>b1</code>, <code>b2</code>, and  <code>b3</code>, and is written in B.py</p></li>
  </ul>
  
  <p>In each of <code>A.py</code> and <code>B.py</code>,  I have a <code>tf.train.Saver</code> that saves the checkpoint of all the local variables, and let's call the checkpoint files <code>ckptA</code> and <code>ckptB</code> respectively.</p>
  
  <p>I now want to make a model C that uses <code>a1</code> and <code>b1</code>. I can make it so that the exact same variable name for <code>a1</code> is used in both A and C by using the var_scope (and the same for <code>b1</code>).</p>
  
  <p>The question is how might I load <code>a1</code> and <code>b1</code> from <code>ckptA</code> and <code>ckptB</code> into model C? For example, would the following work?</p>
  
  <pre><code>saver.restore(session, ckptA_location)
  saver.restore(session, ckptB_location)
  </code></pre>
  
  <p>Would an error be raised if you are try to restore the same session twice? Would it complain that there are no allocated "slots" for the extra variables (<code>b2</code>, <code>b3</code>, <code>a2</code>, <code>a3</code>), or would it simply restore the variables it can, and only complain if there are some other variables in C that are uninitialized?</p>
  
  <p>I'm trying to write some code to test this now but I would love to see a canonical approach to this problem, because one encounters this often when trying to re-use some pre-trained weights.</p>
  
  <p>Thanks!</p>