<p>We have a flask app that's deployed on k8s. The base image of the app is this: <a href="https://hub.docker.com/r/tiangolo/uwsgi-nginx-flask/" rel="nofollow noreferrer">https://hub.docker.com/r/tiangolo/uwsgi-nginx-flask/</a>, and we build our app on top of this. We ship our docker image to ECR, and then deploy pods on k8s.</p>
<p>We want to start running ML models in our k8s nodes. The underlying nodes have GPUs (we're using g4dn instances), and they are using a GPU AMI.</p>
<p>When running our app, I'm seeing the following error:</p>
<pre><code>/usr/local/lib/python3.8/site-packages/torch/cuda/__init__.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at  /pytorch/c10/cuda/CUDAFunctions.cpp:100.)
  return torch._C._cuda_getDeviceCount() &gt; 0
</code></pre>
<p>What's the right way to get CUDA installed on our nodes? I would have expected it to be built into the AMI shipped with the gpu instances but that doesn't seem to be the case.</p>

de1337ed
<p>There are a couple of options:</p>
<ol>
<li>Use <code>tensorflow:latest-gpu</code> as base image and setup additional configuration for your system.</li>
<li><a href="https://www.tensorflow.org/install/gpu" rel="nofollow noreferrer">Setup Cuda drivers</a> yourself in your Docker image.</li>
</ol>
