<p>I am trying to deploy a ML app on Kubernetes engine with GPU. I created the docker image using nvidia/cuda:9.0-runtime and built my app above it. When I deploy the image to Kubernetes Engine I get an error saying that it could not import libcuda.so.1.</p>

<pre><code>ImportError: libcuda.so.1: cannot open shared object file: No such file or directory
</code></pre>

<p>I looked at few solutions posted but none of them seem to work.</p>

<p>When trying those solutions I also found that</p>

<p>the paths mentioned by</p>

<pre><code>echo $LD_LIBRARY_PATH
</code></pre>

<p>which gives </p>

<pre><code>/usr/local/nvidia/lib:/usr/local/nvidia/lib64
</code></pre>

<p>do not seem to exist.</p>

<p>As well as there isn't a file with the name libcuda.so.1 (or any number) anywhere within the file system. And the /usr/lib/cuda/lib64 contains the shared libraries. 
Am I currently implementing anything wrong here</p>

<p>You are facing that issue as you have not installed the CUDA drivers on the cluster. Please follow the installing drivers section in this <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/gpus" rel="nofollow noreferrer">link</a>. For verifying the installation you can run this command and check. </p>

<pre><code>kubectl logs -n kube-system ds/nvidia-driver-installer -c nvidia-driver-installer 
</code></pre>
