title: How to measure GPU memory usage of TensorFlow model
body: <p>My goal is to figure out how much GPU memory a TensorFlow model saved as a .pb file uses during inference. By limiting the <code>per_process_gpu_memory_fraction</code> to a value of 0.01 and successfully running the model on an input image, I would expect a memory usage of 120MB (based on a 12,000MB GPU).</p>
  
  <pre><code>conf = tf.ConfigProto()
  conf.gpu_options.per_process_gpu_memory_fraction=0.01
  session = tf.Session(config=conf)
  </code></pre>
  
  <p>When I observe the actual GPU memory usage via <code>nvidia-smi</code>, I see a utilization of 550MB. Based on the answer here (<a href="https://stackoverflow.com/a/55532954">https://stackoverflow.com/a/55532954</a>), I measure the default memory needed to create a TF session, which amounts to 150MB. Now the question is where the remaining 550MB (measured) - 150MB (session) - 120MB (model) = 280MB are from. I want to exclude any other factors that use the GPU memory and only quantify how much the model itself uses. Can I simply assume 120MB in this case?</p>