title: Tensorflow save model: GraphDef cannot be larger than 2GB
body: <p>I'm getting the following error -- apparently at the time of saving my model</p>
  
  <pre><code>Step = 1799  |  Tensorflow Accuracy = 1.0
  Step = 1799  |  My Accuracy = 0.0363355780022
  Step = 1800  |  Tensorflow Accuracy = 1.0
  Step = 1800  |  My Accuracy = 0.0364694929089
  Traceback (most recent call last):
    File "CNN-LSTM-seg-reg-sigmoid.py", line 290, in &lt;module&gt;
      saver.save(sess, save_path)
    File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1085, in save
      self.export_meta_graph(meta_graph_filename)
    File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1103, in export_meta_graph
      add_shapes=True),
    File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2175, in as_graph_def
      result, _ = self._as_graph_def(from_version, add_shapes)
    File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2138, in _as_graph_def
      raise ValueError("GraphDef cannot be larger than 2GB.")
  ValueError: GraphDef cannot be larger than 2GB.
  </code></pre>
  
  <p><a href="https://groups.google.com/a/tensorflow.org/forum/#!topic/discuss/StRWYvb68T4" rel="nofollow noreferrer">Here</a> suggested to look out for <code>tf.constant</code>s, but I have zero constants in my program. However, my <code>weights</code> and <code>biases</code> are like the following: <code>tf.Variable(tf.random_normal([32]),name="bc1")</code>. Could this be an issue?</p>
  
  <p>If not that, than <a href="https://stackoverflow.com/questions/36349049/overcome-graphdef-cannot-be-larger-than-2gb-in-tensorflow">this</a> tells me that somewhere I am adding to the graph after every loop iteration, but I'm unsure where it is occuring. </p>
  
  <p>My first guess is when I make predictions. I make predictions via the 
  following code...</p>
  
  <pre><code># Make prediction
  im = Image.open('/home/volcart/Documents/Data/input_crops/temp data0001.tif')
  batch_x = np.array(im)
  batch_x = batch_x.reshape((1, n_input_x, n_input_y))
  batch_x = batch_x.astype(float)
  prediction = sess.run(pred, feed_dict={x: batch_x})
  prediction = tf.sigmoid(prediction.reshape((n_input_x * n_input_y, n_classes)))
  prediction = prediction.eval().reshape((n_input_x, n_input_y, n_classes))
  </code></pre>
  
  <p>My second guess is when I calculate <code>loss</code> and <code>accuracy</code> via the following: <code>loss, acc = sess.run([cost, accuracy], feed_dict={x: batch_x, y: batch_y})</code></p>
  
  <p>My entire session code looks like the following:</p>
  
  <pre><code># Initializing the variables
  init = tf.initialize_all_variables()
  saver = tf.train.Saver()
  
  gpu_options = tf.GPUOptions()
  config = tf.ConfigProto(gpu_options=gpu_options)
  config.gpu_options.allow_growth = True
  
  # Launch the graph
  with tf.Session(config=config) as sess:
      sess.run(init)
      summary = tf.train.SummaryWriter('/tmp/logdir/', sess.graph) #initialize graph for tensorboard
      step = 1
      # Import data
      data = scroll_data.read_data('/home/volcart/Documents/Data/')
      # Keep training until reach max iterations
      while step * batch_size &lt; training_iters:
          batch_x, batch_y = data.train.next_batch(batch_size)
          # Run optimization op (backprop)
          batch_x = batch_x.reshape((batch_size, n_input_x, n_input_y))
          batch_y = batch_y.reshape((batch_size, n_input_x, n_input_y))
          batch_y = convert_to_2_channel(batch_y, batch_size)
          sess.run(optimizer, feed_dict={x: batch_x, y: batch_y})
          step = step + 1
  
          loss, acc = sess.run([cost, accuracy], feed_dict={x: batch_x,
                                                            y: batch_y})
  
  
          # Make prediction
          im = Image.open('/home/volcart/Documents/Data/input_crops/temp data0001.tif')
          batch_x = np.array(im)
          batch_x = batch_x.reshape((1, n_input_x, n_input_y))
          batch_x = batch_x.astype(float)
          prediction = sess.run(pred, feed_dict={x: batch_x})
          prediction = tf.sigmoid(prediction.reshape((n_input_x * n_input_y, n_classes)))
          prediction = prediction.eval().reshape((n_input_x, n_input_y, n_classes))
  
          # Temp arrays are to splice the prediction n_input_x x n_input_y x 2
              # into 2 matrices n_input_x x n_input_y
          temp_arr1 = np.empty((n_input_x, n_input_y))
          for i in xrange(n_input_x):
              for j in xrange(n_input_x):
                  for k in xrange(n_classes):
                      if k == 0:
                          temp_arr1[i][j] = 1 - prediction[i][j][k]
  
          my_acc = accuracy_custom(temp_arr1,batch_y[0,:,:,0])
  
          print "Step = " + str(step) + "  |  Tensorflow Accuracy = " + str(acc)
          print "Step = " + str(step) + "  |  My Accuracy = " + str(my_acc)
  
          if step % 100 == 0:
              save_path = "/home/volcart/Documents/CNN-LSTM-reg-model/CNN-LSTM-seg-step-" + str(step) + "-model.ckpt"
              saver.save(sess, save_path)
              csv_file = "/home/volcart/Documents/CNN-LSTM-reg/CNNLSTMreg-step-" + str(step) + "-accuracy-" + str(my_acc) + ".csv"
              np.savetxt(csv_file, temp_arr1, delimiter=",")
  </code></pre>