title: Why is Tensorflow GPU extremely slow when creating models and training models compared to the CPU version?
body: <p>I would first like to give you some information about how I installed tensorflow and other packages before explaining the problem. It took me a lot of time to get tensorflow running on my GPU (Nvidia RTX 3070, Windows 10 system). First, I installed Cuda (v.10.1), downloaded CuDDN (v7.6) and copied and pasted the CuDNN files to the correct Cuda installation folders (as described here: <a href="https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html#install-windows" rel="nofollow noreferrer">https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html#install-windows</a>)
  I want to use tensorflow 2.3.0 and checked if the Cuda and cuDNN versions are compatible using the table on this page: <a href="https://www.tensorflow.org/install/source" rel="nofollow noreferrer">https://www.tensorflow.org/install/source</a>
  Then I opened the anaconda prompt window, activated my new environment (&gt;&gt; activate [MyEnv]) and installed the required packages. I read that it is important to install tensorflow first, so the first package I installed was tensorflow-gpu, followed by a bunch of other packages. Later, I ran into the problem that my GPU was not found when I typed in</p>
  <pre><code>import tensorflow as tf
  print(&quot;Num GPUs Available: &quot;, len(tf.config.list_physical_devices('GPU')))
  </code></pre>
  <p>The response was &quot;Num GPUs Available: 0&quot;
  I did a lot of googling and found the following discussion:
  <a href="https://github.com/ContinuumIO/anaconda-issues/issues/12194#issuecomment-751700156" rel="nofollow noreferrer">https://github.com/ContinuumIO/anaconda-issues/issues/12194#issuecomment-751700156</a>
  where it is mentioned that a faulty tensorflow build is installed when using conda install tensorflow-gpu in the anaconda prompt window. Instead (when using Pythin 3.8, as I do), one has to use pass the correct tensorflow build in the prompt window. So I set up a new environment and used</p>
  <pre><code>conda install tensorflow-gpu=2.3 tensorflow=2.3=mkl_py38h1fcfbd6_0
  </code></pre>
  <p>to install tensorflow. So far so good. Now, the cudatoolkit (version 10.1.243) and cudnn (version 7.6.5), which were missing in my first environment, are inculded in the the tensorflow package and thus in my second environment [MyEnv2].
  I start VSCode, select the correct environment, and retest if the gpu can be found by repeating the test:
  import tensorflow as tf
  print(&quot;Num GPUs Available: &quot;, len(tf.config.list_physical_devices('GPU')))
  And ...it works. The GPU is found and everything looks good at the first sight.</p>
  <p>So what's the problem?
  Using tensorflow on gpu is extremly slow. Not only when training models, but also when creating the model with</p>
  <pre><code>model = models.Sequential()
  model.add(...)
  (...)
  model.summary()
  </code></pre>
  <p>Running the same code sample on CPU finishes almost immediately, wheras running the code on GPU needs more than 10 minutes! (When I look into the taskmanager performance tab nothing happens. Neither CPU nor GPU seems to do anything, when I run the Code on the GPU!) And this happens, when just creating the model without training!
  After compiling the model and starting the training, the same problem occurs. Training on the CPU gives me a immediate feedback about the epoch process, while training on gpu seems to freeze the program as nothing happens for several minutes (maybe &quot;freezing&quot; is the wrong word, because I can still switch between the tabs in VSCode. The program itself is not freezing) Another confusing aspect is that when training on the gpu, I only get nans for the loss and mae when the training finally starts after minutes of waiting. In the task manager I can obeserve that the model needs about 7,5GB of VRAM. The RTX3070 comes with 8GB of VRAM. When I run the same code on the cpu, loss and mae look perfectly fine...</p>
  <p>I really have no idea what is happening and why I am getting this strange behaviour when I run tensorflow on my gpu. Do you have any ideas?</p>