title: nvcc is picking wrong libcudart library
body: <p>This problem comes when, I try to import theano with gpu mode. While importing the theano, it tries to compile some code, make a shared library of it and tries to load it.
  Here is the command to make the so file.</p>
  
  <pre><code>nvcc -shared -O3 -m64 -Xcompiler -DCUDA_NDARRAY_CUH=mc72d035fdf91890f3b36710688069b2e,\
    -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,-fPIC,-fvisibility=hidden -Xlinker \
    -rpath,/home/jay/.theano/compiledir_Linux-4.8--ARCH-x86_64-with-arch-Arch-Linux--3.6.0-64/cuda_ndarray \
    -I/usr/lib/python3.6/site-packages/Theano-0.9.0b1-py3.6.egg/theano/sandbox/cuda \
    -I/usr/lib/python3.6/site-packages/numpy-1.13.0.dev0+72839c4-py3.6-linux-x86_64.egg/numpy/core/include \
    -I/usr/include/python3.6m -I/usr/lib/python3.6/site-packages/Theano-0.9.0b1-py3.6.egg/theano/gof \
    -L/usr/lib -o /home/jay/.theano/compiledir_Linux-4.8--ARCH-x86_64-with-arch-Arch-Linux--3.6.0-64/cuda_ndarray/cuda_ndarray.so \
     mod.cu -lcublas -lpython3.6m 
  </code></pre>
  
  <p>It compiles the successfully but can't find the correct library for <code>cudart</code>.  The <code>ldconfig</code> seems to know the location of this library, which is correct.</p>
  
  <pre><code>$ ldconfig -p | grep libcuda
      libcudart.so.8.0 (libc6,x86-64) =&gt; /opt/cuda/lib64/libcudart.so.8.0
      libcudart.so (libc6,x86-64) =&gt; /opt/cuda/lib64/libcudart.so
      libcuda.so.1 (libc6,x86-64) =&gt; /usr/lib/libcuda.so.1
      libcuda.so (libc6,x86-64) =&gt; /usr/lib/libcuda.so
  </code></pre>
  
  <p>But however when I inspect the library, it states the problem with <code>libcudart</code>.</p>
  
  <pre><code>$ ldd cuda_ndarray.so  | grep cuda
      libcublas.so.8.0 =&gt; /opt/cuda/lib64/libcublas.so.8.0 (0x00007f006dd1b000)
      libcudart.so.7.5 =&gt; not found
  </code></pre>
  
  <p>reading elf header</p>
  
  <pre><code>$ readelf -a cuda_ndarray.so | grep NEEDED
   0x0000000000000001 (NEEDED)             Shared library: [libcublas.so.8.0]
   0x0000000000000001 (NEEDED)             Shared library: [libpython3.6m.so.1.0]
   0x0000000000000001 (NEEDED)             Shared library: [libcudart.so.7.5]
   0x0000000000000001 (NEEDED)             Shared library: [librt.so.1]
   0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
   0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
   0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
   0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
   0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
   0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
  </code></pre>
  
  <p>What went wrong so that it is picking the wrong library, <code>cudart7.5</code> instead of <code>cudart8.0</code> ?</p>
  
  <p>here is a output of my <code>nvcc -V</code></p>
  
  <pre><code>$ nvcc -V
  nvcc: NVIDIA (R) Cuda compiler driver
  Copyright (c) 2005-2016 NVIDIA Corporation
  Built on Sun_Sep__4_22:14:01_CDT_2016
  Cuda compilation tools, release 8.0, V8.0.44
  </code></pre>
  
  <p>and my <code>libcudart.so</code> is pointing to correct library version</p>
  
  <pre><code>$ ls -la | grep libcudart
  lrwxrwxrwx  1 root root        16 Jan 10 06:10 libcudart.so -&gt; libcudart.so.8.0
  lrwxrwxrwx  1 root root        19 Jan 10 06:10 libcudart.so.8.0 -&gt; libcudart.so.8.0.44
  -rwxr-xr-x  1 root root    415432 Jan 10 06:10 libcudart.so.8.0.44
  -rw-r--r--  1 root root    775162 Jan 10 06:10 libcudart_static.a
  </code></pre>
  
  <p>and one more general question how does the linker resolves the actual file location of input like <code>-lm</code> or <code>-lcudart</code> or any shorthand notation used while compiling ?</p>
  
  <p>I tried two of the sample programs shipped with cuda, which includes the <code>libcudart</code> library</p>
  
  <pre><code>$ grep -rnw . -e 'lcudart'
  ./3_Imaging/cudaDecodeGL/Makefile:329:LIBRARIES += -lcudart -lnvcuvid
  ./0_Simple/simpleMPI/Makefile:284:LIBRARIES += -L$(CUDA_PATH)/lib$(LIBSIZE) -lcudart
  </code></pre>
  
  <p>out of these two <code>simpleMPI</code> ran without error.</p>
  
  <pre><code>$ ./simpleMPI 
  Running on 1 nodes
  Average of square roots is: 0.667242
  PASSED
  </code></pre>
  
  <p>The other one failed with earlier error</p>
  
  <pre><code>$ ./cudaDecodeGL 
  ./cudaDecodeGL: error while loading shared libraries: libcudart.so.7.5: cannot open shared object file: No such file or directory
  </code></pre>