title: How do the `intra_op_parallelism_threads` and `inter_op_parallelism_threads` options work in TensorFlow?
body: <p>My understanding is that TensorFlow creates two thread pools on each device: one for intra op parallelism and one for inter op parallelism.</p>
  
  <p>Suppose there are 3 independent ops A, B, C placed on <code>/gpu:0</code> and <code>intra_op_parallelism_threads=5</code>. Suppose A and B have a single-threaded GPU kernel implementation, and C has a multi-threaded kernel implementation, does that mean that they can all run in parallel on the same device, A and B using just 1 GPU thread while C uses up to 3 GPU threads?</p>
  
  <p>Now suppose <code>inter_op_parallelism_threads=2</code>, does that mean that only 2 out of 3 ops can be evaluated simultaneously on <code>/gpu:0</code>, so in the example above, it may be A+B, B+C or A+C depending on who gets there first?</p>
  
  <p>Note: I'm trying to make sense of @mrry's answer to this question: <a href="https://stackoverflow.com/questions/37864081/tensorflow-executing-an-ops-with-a-specific-core-of-a-cpu/37864489#37864489">Tensorflow: executing an ops with a specific core of a CPU</a></p>