title: Selecting the device to be used by a graph in TensorFlow
body: <p>Given a code which uses multiple graphs or multiple versions of the same graph, it is sometimes necessary to ensure that a particular graph uses only CPU for computation, while some other graph uses only GPU.</p>
  
  <p>The basic question is </p>
  
  <blockquote>
    <p><strong>How to make sure that a particular graph makes use of only CPU (XOR) GPU for computations ?</strong></p>
  </blockquote>
  
  <p>There is not an exhaustive discussion of this topic on SO and hence this question.</p>
  
  <p>I have tried a number of different approaches and none seem to work as will be outlined below.</p>
  
  <p>Before further details on the question and various options that have been tried, I will lay down following details :-</p>
  
  <ul>
  <li><strong>TensorFlow Version</strong> : 'v1.1.0-rc2-1003-g3792dd9' 1.1.0-rc2 (<strong>Compiled from source</strong>)</li>
  <li><strong>OS details</strong> : CentOS Linux release 7.2.1511 (Core)</li>
  <li><strong>Bazel version</strong> : 0.4.5</li>
  <li><p>Basic code with which various approaches have been tried is mentioned below :</p>
  
  <pre><code>import tensorflow as tf
  from tensorflow.python.client import timeline
  import matplotlib.pyplot as plt
  
  
  
  def coloraugment(image):
      output = tf.image.random_brightness(image, max_delta=10./255.)
      output = tf.clip_by_value(output, 0.0, 1.0)
      output = tf.image.random_saturation(output, lower=0.5, upper=1.5)
      output = tf.clip_by_value(output, 0.0, 1.0)
      output = tf.image.random_contrast(output, lower=0.5, upper=1.5)
      output = tf.clip_by_value(output, 0.0, 1.0)
      return output
  
  
  def augmentbody(image, sz):
  
      for i in range(10):
          if i == 0:
              cropped = tf.random_crop(value=image, size=sz)
              croppedflipped = tf.image.flip_left_right(cropped)
              out = tf.stack([cropped, croppedflipped], axis=0)
          else:
              cropimg = tf.random_crop(value=image, size=sz)
              augcolor = coloraugment(cropimg)
              augflipped = tf.image.flip_left_right(augcolor)
              coll = tf.stack([augcolor, augflipped], axis=0)
              out = tf.concat([coll, out], axis=0)
  
      out = tf.random_shuffle(out)
      return out
  
  
  def aspect1(aspectratio):
      newheight = tf.constant(256, dtype=tf.float32)
      newwidth = tf.divide(newheight, aspectratio)
      newsize = tf.stack([newheight, newwidth], axis=0)
      newsize = tf.cast(newsize, dtype=tf.int32)
      return newsize
  
  
  def aspect2(aspectratio):
      newwidth = tf.constant(256, dtype=tf.float32)
      newheight = tf.multiply(newwidth, aspectratio)
      newsize = tf.stack([newheight, newwidth], axis=0)
      newsize = tf.cast(newsize, dtype=tf.int32)
      return newsize
  
  
  def resize_image(image):
      imageshape = tf.shape(image)
      imageheight = tf.cast(tf.gather(imageshape, tf.constant(0, dtype=tf.int32)),
                            dtype=tf.float32)
      imagewidth = tf.cast(tf.gather(imageshape, tf.constant(1, dtype=tf.int32)),
                           dtype=tf.float32)
  
      aspectratio = tf.divide(imageheight, imagewidth)
      newsize = tf.cond(tf.less_equal(imageheight, imagewidth),
                        lambda: aspect1(aspectratio),
                        lambda: aspect2(aspectratio))
      image = tf.image.convert_image_dtype(image, dtype=tf.float32)
      image = tf.image.resize_images(image, newsize)
      return image
  
  
  def readimage(file_queue):
      reader = tf.WholeFileReader()
      key, value = reader.read(file_queue)
      image = tf.image.decode_jpeg(value)
      image = resize_image(image)
  
      return image
  
  
  if __name__ == "__main__":
      queue = tf.train.string_input_producer(["holly2.jpg"])
      image = readimage(queue)
      augmented = augmentbody(image, [221,221,3])
      init_op = tf.global_variables_initializer()
      config_cpu = tf.ConfigProto()
      config = tf.ConfigProto(
          device_count = {'GPU': 0}
      )
      sess_cpu = tf.Session(config=config)
      with tf.Session(config=config_cpu) as sess:
          run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)
          run_metadata = tf.RunMetadata()
          coord = tf.train.Coordinator()
          threads = tf.train.start_queue_runners(sess=sess, coord=coord)
          sess.run(init_op)
          [tense] = sess.run([augmented],options=run_options, run_metadata=run_metadata)
          coord.request_stop()
          coord.join(threads)
          tl = timeline.Timeline(run_metadata.step_stats)
          ctf = tl.generate_chrome_trace_format()
          with open('timeline.json', 'w') as f:
              f.write(ctf)
  
      print("The tensor size is {}".format(tense.shape))
      numcols = tense.shape[0]/2
      for i in range(tense.shape[0]):
          plt.subplot(2,numcols,i+1)
          plt.imshow(tense[i, :, :, :])
  
      plt.show()
      plt.close()
  </code></pre></li>
  </ul>
  
  <p><strong>Various approaches which have been tried</strong></p>
  
  <blockquote>
    <p>Various related questions exist on SO with accepted answers, but they
    do not seem to work very well as I outline with examples and outputs.</p>
  </blockquote>
  
  <p><strong>Various tried approaches</strong></p>
  
  <p><strong><em>Approach 1</em></strong></p>
  
  <p>Related question is ( <a href="https://stackoverflow.com/questions/37660312/run-tensorflow-on-cpu">Run Tensorflow on CPU</a> ). The accepted answer is to run <code>tf.Session()</code> with the following configuration :</p>
  
  <pre><code>config = tf.ConfigProto(
          device_count = {'GPU': 0}
      )
  sess = tf.Session(config=config)
  </code></pre>
  
  <p>The corresponding output is :</p>
  
  <pre><code>2017-05-18 13:34:27.477189: I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] Found device 0 with properties: 
  name: GeForce GTX 1080
  major: 6 minor: 1 memoryClockRate (GHz) 1.7715
  pciBusID 0000:04:00.0
  Total memory: 7.92GiB
  Free memory: 7.80GiB
  2017-05-18 13:34:27.477232: I tensorflow/core/common_runtime/gpu/gpu_device.cc:927] DMA: 0 
  2017-05-18 13:34:27.477240: I tensorflow/core/common_runtime/gpu/gpu_device.cc:937] 0:   Y 
  2017-05-18 13:34:27.477259: I tensorflow/core/common_runtime/gpu/gpu_device.cc:996] Creating TensorFlow device (/gpu:0) -&gt; (device: 0, name: GeForce GTX 1080, pci bus id: 0000:04:00.0)
  2017-05-18 13:34:27.482600: I tensorflow/core/common_runtime/gpu/gpu_device.cc:996] Creating TensorFlow device (/gpu:0) -&gt; (device: 0, name: GeForce GTX 1080, pci bus id: 0000:04:00.0)
  2017-05-18 13:34:27.848864: I tensorflow/compiler/xla/service/platform_util.cc:58] platform CUDA present with 1 visible devices
  2017-05-18 13:34:27.848902: I tensorflow/compiler/xla/service/platform_util.cc:58] platform Host present with 40 visible devices
  2017-05-18 13:34:27.851670: I tensorflow/compiler/xla/service/service.cc:184] XLA service 0x7f0fd81d5500 executing computations on platform Host. Devices:
  2017-05-18 13:34:27.851688: I tensorflow/compiler/xla/service/service.cc:192]   StreamExecutor device (0): &lt;undefined&gt;, &lt;undefined&gt;
  2017-05-18 13:34:27.851894: I tensorflow/compiler/xla/service/platform_util.cc:58] platform CUDA present with 1 visible devices
  2017-05-18 13:34:27.851903: I tensorflow/compiler/xla/service/platform_util.cc:58] platform Host present with 40 visible devices
  2017-05-18 13:34:27.854698: I tensorflow/compiler/xla/service/service.cc:184] XLA service 0x7f0fd82b4c50 executing computations on platform CUDA. Devices:
  2017-05-18 13:34:27.854713: I tensorflow/compiler/xla/service/service.cc:192]   StreamExecutor device (0): GeForce GTX 1080, Compute Capability 6.1
  2017-05-18 13:34:28.918980: I tensorflow/stream_executor/dso_loader.cc:139] successfully opened CUDA library libcupti.so.8.0 locally
  </code></pre>
  
  <blockquote>
    <p>You can clearly see that the GPU is still being used and the XLA
    service is running on GPU</p>
  </blockquote>
  
  <p><strong><em>Approach 2</em></strong></p>
  
  <p>Related question is ( <a href="https://stackoverflow.com/questions/37660312/run-tensorflow-on-cpu">Run Tensorflow on CPU</a> ). This <a href="https://stackoverflow.com/a/37663502/6842947">answer</a> states that the following environment variable can be set as follows to use CPU</p>
  
  <pre><code>CUDA_VISIBLE_DEVICES=""
  </code></pre>
  
  <p>When the GPU computation is required, it can be unset.</p>
  
  <p>The corresponding output is </p>
  
  <pre><code>2017-05-18 13:42:24.871020: E tensorflow/stream_executor/cuda/cuda_driver.cc:406] failed call to cuInit: CUDA_ERROR_NO_DEVICE
  2017-05-18 13:42:24.871071: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:158] retrieving CUDA diagnostic information for host: nefgpu12
  2017-05-18 13:42:24.871081: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:165] hostname: nefgpu12
  2017-05-18 13:42:24.871114: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:189] libcuda reported version is: 367.48.0
  2017-05-18 13:42:24.871147: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:369] driver version file contents: """NVRM version: NVIDIA UNIX x86_64 Kernel Module  367.48  Sat Sep  3 18:21:08 PDT 2016
  GCC version:  gcc version 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC) 
  """
  2017-05-18 13:42:24.871170: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:193] kernel reported version is: 367.48.0
  2017-05-18 13:42:24.871178: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:300] kernel version seems to match DSO: 367.48.0
  2017-05-18 13:42:25.159632: W tensorflow/compiler/xla/service/platform_util.cc:61] platform CUDA present but no visible devices found
  2017-05-18 13:42:25.159674: I tensorflow/compiler/xla/service/platform_util.cc:58] platform Host present with 40 visible devices
  2017-05-18 13:42:25.162626: I tensorflow/compiler/xla/service/service.cc:184] XLA service 0x7f5798002df0 executing computations on platform Host. Devices:
  2017-05-18 13:42:25.162663: I tensorflow/compiler/xla/service/service.cc:192]   StreamExecutor device (0): &lt;undefined&gt;, &lt;undefined&gt;
  2017-05-18 13:42:25.223309: I tensorflow/stream_executor/dso_loader.cc:139] successfully opened CUDA library libcupti.so.8.0 locally
  </code></pre>
  
  <blockquote>
    <p>You can see from this output that the GPU is not being used.</p>
  </blockquote>
  
  <p><strong><em>Approach 3</em></strong></p>
  
  <p>The related question is ( <a href="https://stackoverflow.com/questions/43748726/running-multiple-graphs-in-different-device-modes-in-tensorflow">Running multiple graphs in different device modes in TensorFlow</a> ). One answer gives the following solution :</p>
  
  <pre><code># The config for CPU usage
  config_cpu = tf.ConfigProto()
  config_cpu.gpu_options.visible_device_list=''
  sess_cpu = tf.Session(config=config_cpu)
  
  # The config for GPU usage
  config_gpu = tf.ConfigProto()
  config_gpu.gpu_options.visible_device_list='0'
  sess_gpu = tf.Session(config=config_gpu)
  </code></pre>
  
  <p>The output of using the configuration for CPU usage as outlined in the solution is as follows :</p>
  
  <pre><code>2017-05-18 13:50:32.999431: I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] Found device 0 with properties: 
  name: GeForce GTX 1080
  major: 6 minor: 1 memoryClockRate (GHz) 1.7715
  pciBusID 0000:04:00.0
  Total memory: 7.92GiB
  Free memory: 7.80GiB
  2017-05-18 13:50:32.999472: I tensorflow/core/common_runtime/gpu/gpu_device.cc:927] DMA: 0 
  2017-05-18 13:50:32.999478: I tensorflow/core/common_runtime/gpu/gpu_device.cc:937] 0:   Y 
  2017-05-18 13:50:32.999490: I tensorflow/core/common_runtime/gpu/gpu_device.cc:996] Creating TensorFlow device (/gpu:0) -&gt; (device: 0, name: GeForce GTX 1080, pci bus id: 0000:04:00.0)
  2017-05-18 13:50:33.084737: I tensorflow/core/common_runtime/gpu/gpu_device.cc:996] Creating TensorFlow device (/gpu:0) -&gt; (device: 0, name: GeForce GTX 1080, pci bus id: 0000:04:00.0)
  2017-05-18 13:50:33.395798: I tensorflow/compiler/xla/service/platform_util.cc:58] platform CUDA present with 1 visible devices
  2017-05-18 13:50:33.395837: I tensorflow/compiler/xla/service/platform_util.cc:58] platform Host present with 40 visible devices
  2017-05-18 13:50:33.398634: I tensorflow/compiler/xla/service/service.cc:184] XLA service 0x7f08181ecfa0 executing computations on platform Host. Devices:
  2017-05-18 13:50:33.398695: I tensorflow/compiler/xla/service/service.cc:192]   StreamExecutor device (0): &lt;undefined&gt;, &lt;undefined&gt;
  2017-05-18 13:50:33.398908: I tensorflow/compiler/xla/service/platform_util.cc:58] platform CUDA present with 1 visible devices
  2017-05-18 13:50:33.398920: I tensorflow/compiler/xla/service/platform_util.cc:58] platform Host present with 40 visible devices
  2017-05-18 13:50:33.401731: I tensorflow/compiler/xla/service/service.cc:184] XLA service 0x7f081821e1f0 executing computations on platform CUDA. Devices:
  2017-05-18 13:50:33.401745: I tensorflow/compiler/xla/service/service.cc:192]   StreamExecutor device (0): GeForce GTX 1080, Compute Capability 6.1
  2017-05-18 13:50:34.484142: I tensorflow/stream_executor/dso_loader.cc:139] successfully opened CUDA library libcupti.so.8.0 locally
  </code></pre>
  
  <blockquote>
    <p>You can see that the GPU is still being used.</p>
  </blockquote>