title: Tensorflow with GPU slower than expected
body: <p>So I recently tried running tensorflow-gpu on a pc with the following specs:</p>
  <p>AMD Ryzen 5 2600X 6 core, NVIDIA GeForce RTX 2060 with 16 GB ram</p>
  <p>I ran the built in dataset with Fashion mnist in the tutorial on <a href="https://colab.research.google.com/github/lmoroney/mlday-tokyo/blob/master/Lab2-Computer-Vision.ipynb" rel="noreferrer">colab</a>. I ran the following code and noticed that colab did not run on a gpu:</p>
  <p><code>print(&quot;GPU is&quot;, &quot;available&quot; if tf.config.list_physical_devices('GPU') else &quot;NOT AVAILABLE&quot;)</code></p>
  <p>So I went through the tutorial and essentially ran their code:</p>
  <pre><code>import tensorflow as tf
  import time
  
  print(&quot;GPU is&quot;, &quot;available&quot; if tf.config.list_physical_devices('GPU') else &quot;NOT AVAILABLE&quot;)
  
  mnist = tf.keras.datasets.fashion_mnist
  (training_images, training_labels), (test_images, test_labels) = mnist.load_data()
  training_images  = training_images / 255.0
  test_images = test_images / 255.0
  
  model = tf.keras.models.Sequential([tf.keras.layers.Flatten(), 
                                      tf.keras.layers.Dense(128, activation=tf.nn.relu), 
                                      tf.keras.layers.Dense(10, activation=tf.nn.softmax)])
  
  
  
  start_time = time.time()
  model.compile(optimizer = tf.keras.optimizers.Adam(),
                loss = 'sparse_categorical_crossentropy',
                metrics=['accuracy'])
  model.fit(training_images, training_labels, epochs=5)
  
  print(&quot;My program took {} seconds to run&quot;.format(time.time() - start_time))
  </code></pre>
  <p>I noticed that on colab it took ~17 seconds to compile and fit the data. When I ran it on my computer which did detect the GPU, it took ~13 seconds to do the same process. I was under the impression that the GPU I had would be light years faster so I was wondering what the problem is with my setup or if I was using my GPU incorrectly.</p>
  <p>Also I am running python 3.7.7, tensorflow version 2.1.0, and keras version 2.2.4-tf.</p>