<p dir="auto"><strong>System information</strong></p>
<ul dir="auto">
<li>Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes</li>
<li>OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Win 10</li>
<li>TensorFlow installed from (source or binary): pip</li>
<li>TensorFlow version (use command below): 2.0a-gpu</li>
<li>Python version: 3.7.3</li>
<li>CUDA/cuDNN version: 10.0</li>
<li>GPU model and memory: RTX 2060</li>
</ul>
<p dir="auto"><strong>Describe the current behavior</strong></p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import tensorflow as tf
from tensorflow.python.ops import linalg_ops

mat = tf.random.uniform([2048, 2048])

s, u, v = linalg_ops.svd(mat)"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">tensorflow</span> <span class="pl-k">as</span> <span class="pl-s1">tf</span>
<span class="pl-k">from</span> <span class="pl-s1">tensorflow</span>.<span class="pl-s1">python</span>.<span class="pl-s1">ops</span> <span class="pl-k">import</span> <span class="pl-s1">linalg_ops</span>

<span class="pl-s1">mat</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-s1">random</span>.<span class="pl-en">uniform</span>([<span class="pl-c1">2048</span>, <span class="pl-c1">2048</span>])

<span class="pl-s1">s</span>, <span class="pl-s1">u</span>, <span class="pl-s1">v</span> <span class="pl-c1">=</span> <span class="pl-s1">linalg_ops</span>.<span class="pl-en">svd</span>(<span class="pl-s1">mat</span>)</pre></div>
<p dir="auto">When running this example with my tensorflow-gpu installation it hangs (until I cancel with ctrl+c, which takes very long):</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="2019-05-08 13:20:05.994717: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2019-05-08 13:20:06.002398: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library nvcuda.dll
2019-05-08 13:20:06.250271: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1467] Found device 0 with properties:
name: GeForce RTX 2060 major: 7 minor: 5 memoryClockRate(GHz): 1.71
pciBusID: 0000:1f:00.0
totalMemory: 6.00GiB freeMemory: 4.89GiB
2019-05-08 13:20:06.256846: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1546] Adding visible gpu devices: 0
2019-05-08 13:20:06.803341: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1015] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-05-08 13:20:06.806295: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1021]      0
2019-05-08 13:20:06.810187: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1034] 0:   N
2019-05-08 13:20:06.812341: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1149] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4618 MB memory) -&gt; physical GPU (device: 0,
name: GeForce RTX 2060, pci bus id: 0000:1f:00.0, compute capability: 7.5)
2019-05-08 13:20:07.023250: I tensorflow/core/kernels/cuda_solvers.cc:159] Creating CudaSolver handles
for stream 000001C2BFA888A0
Traceback (most recent call last):
  File &quot;.\svd_check.py&quot;, line 6, in &lt;module&gt;
    s, u, v = linalg_ops.svd(mat)
  File &quot;C:\Progams\Miniconda\envs\tf2-gpu\lib\site-packages\tensorflow\python\ops\linalg_ops.py&quot;, line 418, in svd
    tensor, compute_uv=compute_uv, full_matrices=full_matrices, name=name)
  File &quot;C:\Progams\Miniconda\envs\tf2-gpu\lib\site-packages\tensorflow\python\ops\gen_linalg_ops.py&quot;, line 2409, in svd
    &quot;full_matrices&quot;, full_matrices)
KeyboardInterrupt"><pre class="notranslate"><code class="notranslate">2019-05-08 13:20:05.994717: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2019-05-08 13:20:06.002398: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library nvcuda.dll
2019-05-08 13:20:06.250271: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1467] Found device 0 with properties:
name: GeForce RTX 2060 major: 7 minor: 5 memoryClockRate(GHz): 1.71
pciBusID: 0000:1f:00.0
totalMemory: 6.00GiB freeMemory: 4.89GiB
2019-05-08 13:20:06.256846: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1546] Adding visible gpu devices: 0
2019-05-08 13:20:06.803341: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1015] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-05-08 13:20:06.806295: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1021]      0
2019-05-08 13:20:06.810187: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1034] 0:   N
2019-05-08 13:20:06.812341: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1149] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4618 MB memory) -&gt; physical GPU (device: 0,
name: GeForce RTX 2060, pci bus id: 0000:1f:00.0, compute capability: 7.5)
2019-05-08 13:20:07.023250: I tensorflow/core/kernels/cuda_solvers.cc:159] Creating CudaSolver handles
for stream 000001C2BFA888A0
Traceback (most recent call last):
  File ".\svd_check.py", line 6, in &lt;module&gt;
    s, u, v = linalg_ops.svd(mat)
  File "C:\Progams\Miniconda\envs\tf2-gpu\lib\site-packages\tensorflow\python\ops\linalg_ops.py", line 418, in svd
    tensor, compute_uv=compute_uv, full_matrices=full_matrices, name=name)
  File "C:\Progams\Miniconda\envs\tf2-gpu\lib\site-packages\tensorflow\python\ops\gen_linalg_ops.py", line 2409, in svd
    "full_matrices", full_matrices)
KeyboardInterrupt
</code></pre></div>
<p dir="auto">On tensorflow most recent v1 version it just works.</p>
<p dir="auto">Also, when running the following it works as well:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import tensorflow as tf
from tensorflow.python.ops import linalg_ops

mat = tf.random.uniform([2048, 2048])

with tf.device('/device:CPU:0'):
    s, u, v = linalg_ops.svd(mat)"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">tensorflow</span> <span class="pl-k">as</span> <span class="pl-s1">tf</span>
<span class="pl-k">from</span> <span class="pl-s1">tensorflow</span>.<span class="pl-s1">python</span>.<span class="pl-s1">ops</span> <span class="pl-k">import</span> <span class="pl-s1">linalg_ops</span>

<span class="pl-s1">mat</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-s1">random</span>.<span class="pl-en">uniform</span>([<span class="pl-c1">2048</span>, <span class="pl-c1">2048</span>])

<span class="pl-k">with</span> <span class="pl-s1">tf</span>.<span class="pl-en">device</span>(<span class="pl-s">'/device:CPU:0'</span>):
    <span class="pl-s1">s</span>, <span class="pl-s1">u</span>, <span class="pl-s1">v</span> <span class="pl-c1">=</span> <span class="pl-s1">linalg_ops</span>.<span class="pl-en">svd</span>(<span class="pl-s1">mat</span>)</pre></div>
<p dir="auto">OS:</p>
<ul dir="auto">
<li><strong>OS Platform and Distribution (e.g., Linux Ubuntu 16.04)</strong>: CentOS release 7.4.1708</li>
<li><strong>TensorFlow installed from (source or binary)</strong>: From source</li>
<li><strong>Python version</strong>: 2.7.13</li>
<li><strong>Bazel version</strong>: 0.6.1</li>
<li><strong>CUDA/cuDNN version</strong>: CUDA 8.0/cuDNN 6.0.21</li>
<li><strong>GPU model and memory</strong>: GeForce GTX 950M, memory 4GB</li>
</ul>
<p dir="auto">output of <code class="notranslate">tf_env_collect.sh</code></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="
== cat /etc/issue ===============================================
Linux zhanghao 3.10.0-693.2.2.el7.x86_64 #1 SMP Tue Sep 12 22:26:13 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
VERSION=&quot;7 (Core)&quot;
VERSION_ID=&quot;7&quot;
CENTOS_MANTISBT_PROJECT_VERSION=&quot;7&quot;
REDHAT_SUPPORT_PRODUCT_VERSION=&quot;7&quot;

== are we in docker =============================================
No

== compiler =====================================================
c++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-16)
Copyright © 2015 Free Software Foundation, Inc.
本程序是自由软件；请参看源代码的版权声明。本软件没有任何担保；
包括没有适销性和某一专用目的下的适用性担保。

== uname -a =====================================================
Linux zhanghao 3.10.0-693.2.2.el7.x86_64 #1 SMP Tue Sep 12 22:26:13 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

== check pips ===================================================

== check for virtualenv =========================================
False

== tensorflow import ============================================
Traceback (most recent call last):
  File &quot;&lt;string&gt;&quot;, line 1, in &lt;module&gt;
ImportError: No module named tensorflow

== env ==========================================================
LD_LIBRARY_PATH is unset
DYLD_LIBRARY_PATH is unset

== nvidia-smi ===================================================
Tue Oct 10 16:36:08 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.90                 Driver Version: 384.90                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 950M    Off  | 00000000:0A:00.0 Off |                  N/A |
| N/A   45C    P0    N/A /  N/A |      0MiB /  4044MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

== cuda libs  ===================================================

== cat /etc/issue ===============================================
Linux zhanghao 3.10.0-693.2.2.el7.x86_64 #1 SMP Tue Sep 12 22:26:13 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
VERSION=&quot;7 (Core)&quot;
VERSION_ID=&quot;7&quot;
CENTOS_MANTISBT_PROJECT_VERSION=&quot;7&quot;
REDHAT_SUPPORT_PRODUCT_VERSION=&quot;7&quot;

== are we in docker =============================================
No

== compiler =====================================================
c++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-16)
Copyright © 2015 Free Software Foundation, Inc.
本程序是自由软件；请参看源代码的版权声明。本软件没有任何担保；
包括没有适销性和某一专用目的下的适用性担保。

== uname -a =====================================================
Linux zhanghao 3.10.0-693.2.2.el7.x86_64 #1 SMP Tue Sep 12 22:26:13 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

== check pips ===================================================
numpy (1.12.1)
protobuf (3.4.0)
tensorflow (1.4.0rc0)
tensorflow-tensorboard (0.4.0rc1)

== check for virtualenv =========================================
False

== tensorflow import ============================================
tf.VERSION = 1.4.0-rc0
tf.GIT_VERSION = v1.3.0-rc1-3111-g4196d6d
tf.COMPILER_VERSION = v1.3.0-rc1-3111-g4196d6d
Sanity check: array([1], dtype=int32)

== env ==========================================================
LD_LIBRARY_PATH /usr/local/cuda/lib64/:/usr/local/cuda/lib64/stubs/:/usr/local/cuda/extras/CUPTI/lib64/:/usr/local/cuda/nvvm/lib64/:/usr/lib64/nvidia/:/opt/intel/compilers_and_libraries_2017.4.196/linux/compiler/lib/intel64:/opt/intel/compilers_and_libraries_2017.4.196/linux/compiler/lib/intel64_lin:/opt/intel/compilers_and_libraries_2017.4.196/linux/mpi/intel64/lib:/opt/intel/compilers_and_libraries_2017.4.196/linux/mpi/mic/lib:/opt/intel/compilers_and_libraries_2017.4.196/linux/ipp/lib/intel64:/opt/intel/compilers_and_libraries_2017.4.196/linux/compiler/lib/intel64_lin:/opt/intel/compilers_and_libraries_2017.4.196/linux/mkl/lib/intel64_lin:/opt/intel/compilers_and_libraries_2017.4.196/linux/tbb/lib/intel64/gcc4.7:/opt/intel/debugger_2017/iga/lib:/opt/intel/debugger_2017/libipt/intel64/lib:/opt/intel/compilers_and_libraries_2017.4.196/linux/daal/lib/intel64_lin:/opt/intel/compilers_and_libraries_2017.4.196/linux/daal/../tbb/lib/intel64_lin/gcc4.4
DYLD_LIBRARY_PATH is unset

== nvidia-smi ===================================================
Tue Oct 10 16:36:37 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.90                 Driver Version: 384.90                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 950M    Off  | 00000000:0A:00.0 Off |                  N/A |
| N/A   45C    P0    N/A /  N/A |      0MiB /  4044MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

== cuda libs  ===================================================
/usr/local/cuda-8.0/doc/man/man7/libcudart.so.7
/usr/local/cuda-8.0/doc/man/man7/libcudart.7
/usr/local/cuda-8.0/targets/x86_64-linux/lib/libcudart.so.8.0.61
/usr/local/cuda-8.0/targets/x86_64-linux/lib/libcudart_static.a"><pre class="notranslate"><code class="notranslate">
== cat /etc/issue ===============================================
Linux zhanghao 3.10.0-693.2.2.el7.x86_64 #1 SMP Tue Sep 12 22:26:13 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
VERSION="7 (Core)"
VERSION_ID="7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

== are we in docker =============================================
No

== compiler =====================================================
c++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-16)
Copyright © 2015 Free Software Foundation, Inc.
本程序是自由软件；请参看源代码的版权声明。本软件没有任何担保；
包括没有适销性和某一专用目的下的适用性担保。

== uname -a =====================================================
Linux zhanghao 3.10.0-693.2.2.el7.x86_64 #1 SMP Tue Sep 12 22:26:13 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

== check pips ===================================================

== check for virtualenv =========================================
False

== tensorflow import ============================================
Traceback (most recent call last):
  File "&lt;string&gt;", line 1, in &lt;module&gt;
ImportError: No module named tensorflow

== env ==========================================================
LD_LIBRARY_PATH is unset
DYLD_LIBRARY_PATH is unset

== nvidia-smi ===================================================
Tue Oct 10 16:36:08 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.90                 Driver Version: 384.90                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 950M    Off  | 00000000:0A:00.0 Off |                  N/A |
| N/A   45C    P0    N/A /  N/A |      0MiB /  4044MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

== cuda libs  ===================================================

== cat /etc/issue ===============================================
Linux zhanghao 3.10.0-693.2.2.el7.x86_64 #1 SMP Tue Sep 12 22:26:13 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
VERSION="7 (Core)"
VERSION_ID="7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

== are we in docker =============================================
No

== compiler =====================================================
c++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-16)
Copyright © 2015 Free Software Foundation, Inc.
本程序是自由软件；请参看源代码的版权声明。本软件没有任何担保；
包括没有适销性和某一专用目的下的适用性担保。

== uname -a =====================================================
Linux zhanghao 3.10.0-693.2.2.el7.x86_64 #1 SMP Tue Sep 12 22:26:13 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

== check pips ===================================================
numpy (1.12.1)
protobuf (3.4.0)
tensorflow (1.4.0rc0)
tensorflow-tensorboard (0.4.0rc1)

== check for virtualenv =========================================
False

== tensorflow import ============================================
tf.VERSION = 1.4.0-rc0
tf.GIT_VERSION = v1.3.0-rc1-3111-g4196d6d
tf.COMPILER_VERSION = v1.3.0-rc1-3111-g4196d6d
Sanity check: array([1], dtype=int32)

== env ==========================================================
LD_LIBRARY_PATH /usr/local/cuda/lib64/:/usr/local/cuda/lib64/stubs/:/usr/local/cuda/extras/CUPTI/lib64/:/usr/local/cuda/nvvm/lib64/:/usr/lib64/nvidia/:/opt/intel/compilers_and_libraries_2017.4.196/linux/compiler/lib/intel64:/opt/intel/compilers_and_libraries_2017.4.196/linux/compiler/lib/intel64_lin:/opt/intel/compilers_and_libraries_2017.4.196/linux/mpi/intel64/lib:/opt/intel/compilers_and_libraries_2017.4.196/linux/mpi/mic/lib:/opt/intel/compilers_and_libraries_2017.4.196/linux/ipp/lib/intel64:/opt/intel/compilers_and_libraries_2017.4.196/linux/compiler/lib/intel64_lin:/opt/intel/compilers_and_libraries_2017.4.196/linux/mkl/lib/intel64_lin:/opt/intel/compilers_and_libraries_2017.4.196/linux/tbb/lib/intel64/gcc4.7:/opt/intel/debugger_2017/iga/lib:/opt/intel/debugger_2017/libipt/intel64/lib:/opt/intel/compilers_and_libraries_2017.4.196/linux/daal/lib/intel64_lin:/opt/intel/compilers_and_libraries_2017.4.196/linux/daal/../tbb/lib/intel64_lin/gcc4.4
DYLD_LIBRARY_PATH is unset

== nvidia-smi ===================================================
Tue Oct 10 16:36:37 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.90                 Driver Version: 384.90                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 950M    Off  | 00000000:0A:00.0 Off |                  N/A |
| N/A   45C    P0    N/A /  N/A |      0MiB /  4044MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

== cuda libs  ===================================================
/usr/local/cuda-8.0/doc/man/man7/libcudart.so.7
/usr/local/cuda-8.0/doc/man/man7/libcudart.7
/usr/local/cuda-8.0/targets/x86_64-linux/lib/libcudart.so.8.0.61
/usr/local/cuda-8.0/targets/x86_64-linux/lib/libcudart_static.a
</code></pre></div>
<p dir="auto">output of <code class="notranslate">python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"</code></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="('v1.3.0-rc1-3111-g4196d6d', '1.4.0-rc0')"><pre class="notranslate"><code class="notranslate">('v1.3.0-rc1-3111-g4196d6d', '1.4.0-rc0')
</code></pre></div>
<h3 dir="auto">Describe the problem</h3>
<p dir="auto">SVD on GPU is slower than SVD on CPU</p>
<h3 dir="auto">Source code / logs</h3>
<p dir="auto">file main.py</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import tensorflow as tf
import numpy as np
import sys

D = 1024
dA = np.random.normal(size=(D,D))

dev = &quot;/gpu:0&quot; if len(sys.argv)==1 else &quot;/cpu:0&quot;

with tf.device(dev):
    A = tf.placeholder(shape=(D,D),dtype=tf.float32)
    S, U, V = tf.svd(A)

config = tf.ConfigProto()
config.log_device_placement = True
config.graph_options.optimizer_options.global_jit_level=tf.OptimizerOptions.ON_1
sess = tf.Session(config=config)

for _ in xrange(10):
    dS, dU, dV = sess.run((S, U, V), feed_dict={A:dA})"><pre class="notranslate"><code class="notranslate">import tensorflow as tf
import numpy as np
import sys

D = 1024
dA = np.random.normal(size=(D,D))

dev = "/gpu:0" if len(sys.argv)==1 else "/cpu:0"

with tf.device(dev):
    A = tf.placeholder(shape=(D,D),dtype=tf.float32)
    S, U, V = tf.svd(A)

config = tf.ConfigProto()
config.log_device_placement = True
config.graph_options.optimizer_options.global_jit_level=tf.OptimizerOptions.ON_1
sess = tf.Session(config=config)

for _ in xrange(10):
    dS, dU, dV = sess.run((S, U, V), feed_dict={A:dA})
</code></pre></div>
<h2 dir="auto">run on GPU</h2>
<p dir="auto"><code class="notranslate">time python main.py</code></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="2017-10-10 16:28:49.047703: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:892] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2017-10-10 16:28:49.048176: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties:
name: GeForce GTX 950M major: 5 minor: 0 memoryClockRate(GHz): 1.124
pciBusID: 0000:0a:00.0
totalMemory: 3.95GiB freeMemory: 3.91GiB
2017-10-10 16:28:49.048205: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -&gt; (device: 0, name: GeForce GTX 950M, pci bus id: 0000:0a:00.0, compute capability: 5.0)
Device mapping:
/job:localhost/replica:0/task:0/device:XLA_GPU:0 -&gt; device: XLA_GPU device
/job:localhost/replica:0/task:0/device:XLA_CPU:0 -&gt; device: XLA_CPU device
/job:localhost/replica:0/task:0/device:GPU:0 -&gt; device: 0, name: GeForce GTX 950M, pci bus id: 0000:0a:00.0, compute capability: 5.0
2017-10-10 16:28:49.064960: I tensorflow/core/common_runtime/direct_session.cc:299] Device mapping:
/job:localhost/replica:0/task:0/device:XLA_GPU:0 -&gt; device: XLA_GPU device
/job:localhost/replica:0/task:0/device:XLA_CPU:0 -&gt; device: XLA_CPU device
/job:localhost/replica:0/task:0/device:GPU:0 -&gt; device: 0, name: GeForce GTX 950M, pci bus id: 0000:0a:00.0, compute capability: 5.0

Svd: (Svd): /job:localhost/replica:0/task:0/device:GPU:0
2017-10-10 16:28:49.067234: I tensorflow/core/common_runtime/placer.cc:874] Svd: (Svd)/job:localhost/replica:0/task:0/device:GPU:0
Placeholder: (Placeholder): /job:localhost/replica:0/task:0/device:GPU:0
2017-10-10 16:28:49.067302: I tensorflow/core/common_runtime/placer.cc:874] Placeholder: (Placeholder)/job:localhost/replica:0/task:0/device:GPU:0
2017-10-10 16:28:49.074053: I tensorflow/core/kernels/cuda_solvers.cc:159] Creating CudaSolver handles for stream 0x488e860
python main.py  27.50s user 2.30s system 100% cpu 29.658 total"><pre class="notranslate"><code class="notranslate">2017-10-10 16:28:49.047703: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:892] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2017-10-10 16:28:49.048176: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties:
name: GeForce GTX 950M major: 5 minor: 0 memoryClockRate(GHz): 1.124
pciBusID: 0000:0a:00.0
totalMemory: 3.95GiB freeMemory: 3.91GiB
2017-10-10 16:28:49.048205: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -&gt; (device: 0, name: GeForce GTX 950M, pci bus id: 0000:0a:00.0, compute capability: 5.0)
Device mapping:
/job:localhost/replica:0/task:0/device:XLA_GPU:0 -&gt; device: XLA_GPU device
/job:localhost/replica:0/task:0/device:XLA_CPU:0 -&gt; device: XLA_CPU device
/job:localhost/replica:0/task:0/device:GPU:0 -&gt; device: 0, name: GeForce GTX 950M, pci bus id: 0000:0a:00.0, compute capability: 5.0
2017-10-10 16:28:49.064960: I tensorflow/core/common_runtime/direct_session.cc:299] Device mapping:
/job:localhost/replica:0/task:0/device:XLA_GPU:0 -&gt; device: XLA_GPU device
/job:localhost/replica:0/task:0/device:XLA_CPU:0 -&gt; device: XLA_CPU device
/job:localhost/replica:0/task:0/device:GPU:0 -&gt; device: 0, name: GeForce GTX 950M, pci bus id: 0000:0a:00.0, compute capability: 5.0

Svd: (Svd): /job:localhost/replica:0/task:0/device:GPU:0
2017-10-10 16:28:49.067234: I tensorflow/core/common_runtime/placer.cc:874] Svd: (Svd)/job:localhost/replica:0/task:0/device:GPU:0
Placeholder: (Placeholder): /job:localhost/replica:0/task:0/device:GPU:0
2017-10-10 16:28:49.067302: I tensorflow/core/common_runtime/placer.cc:874] Placeholder: (Placeholder)/job:localhost/replica:0/task:0/device:GPU:0
2017-10-10 16:28:49.074053: I tensorflow/core/kernels/cuda_solvers.cc:159] Creating CudaSolver handles for stream 0x488e860
python main.py  27.50s user 2.30s system 100% cpu 29.658 total
</code></pre></div>
<h2 dir="auto">run on CPU</h2>
<p dir="auto"><code class="notranslate">time python main.py -</code></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="2017-10-10 16:29:53.252138: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:892] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2017-10-10 16:29:53.252572: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties:
name: GeForce GTX 950M major: 5 minor: 0 memoryClockRate(GHz): 1.124
pciBusID: 0000:0a:00.0
totalMemory: 3.95GiB freeMemory: 3.91GiB
2017-10-10 16:29:53.252600: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -&gt; (device: 0, name: GeForce GTX 950M, pci bus id: 0000:0a:00.0, compute capability: 5.0)
Device mapping:
/job:localhost/replica:0/task:0/device:XLA_GPU:0 -&gt; device: XLA_GPU device
/job:localhost/replica:0/task:0/device:XLA_CPU:0 -&gt; device: XLA_CPU device
/job:localhost/replica:0/task:0/device:GPU:0 -&gt; device: 0, name: GeForce GTX 950M, pci bus id: 0000:0a:00.0, compute capability: 5.0
2017-10-10 16:29:53.269242: I tensorflow/core/common_runtime/direct_session.cc:299] Device mapping:
/job:localhost/replica:0/task:0/device:XLA_GPU:0 -&gt; device: XLA_GPU device
/job:localhost/replica:0/task:0/device:XLA_CPU:0 -&gt; device: XLA_CPU device
/job:localhost/replica:0/task:0/device:GPU:0 -&gt; device: 0, name: GeForce GTX 950M, pci bus id: 0000:0a:00.0, compute capability: 5.0

Svd: (Svd): /job:localhost/replica:0/task:0/device:CPU:0
2017-10-10 16:29:53.271505: I tensorflow/core/common_runtime/placer.cc:874] Svd: (Svd)/job:localhost/replica:0/task:0/device:CPU:0
Placeholder: (Placeholder): /job:localhost/replica:0/task:0/device:CPU:0
2017-10-10 16:29:53.271544: I tensorflow/core/common_runtime/placer.cc:874] Placeholder: (Placeholder)/job:localhost/replica:0/task:0/device:CPU:0
python main.py -  34.33s user 10.68s system 621% cpu 7.241 total"><pre class="notranslate"><code class="notranslate">2017-10-10 16:29:53.252138: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:892] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2017-10-10 16:29:53.252572: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties:
name: GeForce GTX 950M major: 5 minor: 0 memoryClockRate(GHz): 1.124
pciBusID: 0000:0a:00.0
totalMemory: 3.95GiB freeMemory: 3.91GiB
2017-10-10 16:29:53.252600: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -&gt; (device: 0, name: GeForce GTX 950M, pci bus id: 0000:0a:00.0, compute capability: 5.0)
Device mapping:
/job:localhost/replica:0/task:0/device:XLA_GPU:0 -&gt; device: XLA_GPU device
/job:localhost/replica:0/task:0/device:XLA_CPU:0 -&gt; device: XLA_CPU device
/job:localhost/replica:0/task:0/device:GPU:0 -&gt; device: 0, name: GeForce GTX 950M, pci bus id: 0000:0a:00.0, compute capability: 5.0
2017-10-10 16:29:53.269242: I tensorflow/core/common_runtime/direct_session.cc:299] Device mapping:
/job:localhost/replica:0/task:0/device:XLA_GPU:0 -&gt; device: XLA_GPU device
/job:localhost/replica:0/task:0/device:XLA_CPU:0 -&gt; device: XLA_CPU device
/job:localhost/replica:0/task:0/device:GPU:0 -&gt; device: 0, name: GeForce GTX 950M, pci bus id: 0000:0a:00.0, compute capability: 5.0

Svd: (Svd): /job:localhost/replica:0/task:0/device:CPU:0
2017-10-10 16:29:53.271505: I tensorflow/core/common_runtime/placer.cc:874] Svd: (Svd)/job:localhost/replica:0/task:0/device:CPU:0
Placeholder: (Placeholder): /job:localhost/replica:0/task:0/device:CPU:0
2017-10-10 16:29:53.271544: I tensorflow/core/common_runtime/placer.cc:874] Placeholder: (Placeholder)/job:localhost/replica:0/task:0/device:CPU:0
python main.py -  34.33s user 10.68s system 621% cpu 7.241 total
</code></pre></div>