title: Pre-train neural networks in TensorFlow
body: <p>I'm training a deep neural network. Training directly the full network is difficult, therefore, I prefer to train layer by layer. Firstly, I train a network with a single hidden layer. After that, I save the model using <code>tf.train.Saver</code>. Next time, I restore variables by calling: </p>
  
  <pre><code>saver.restore(sess, "runs/simple-model.ckpt")
  </code></pre>
  
  <p>Of course, this time, the architecture has changed because I insert a new layer between the hidden layer and the output layer, and a new variable is used. For this reason, Tensorflow throws an error: <code>Key not found</code></p>
  
  <p>My question is that: How to train neural network layer by layer in Tensorflow? I want to do it before final fine-tuning of the full network.
  Many thanks.</p>