<p>I am running GPU instance on GKE when everything is deployed I make the request to the service Above mention error occur
I followed all the step in mentioned in <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/gpus#ubuntu" rel="nofollow noreferrer">https://cloud.google.com/kubernetes-engine/docs/how-to/gpus#ubuntu</a>
This is my DockerFile</p>
<pre><code>FROM nvidia/cuda:10.2-cudnn7-devel

# install nginx
# RUN apt-get update &amp;&amp; apt-get install nginx vim -y --no-install-recommends
# RUN ln -sf /dev/stdout /var/log/nginx/access.log \
#     &amp;&amp; ln -sf /dev/stderr /var/log/nginx/error.log

## Setup 

RUN mkdir -p /opt/app

RUN apt-get update -y &amp;&amp; \
    apt-get install -y --no-install-recommends \
    python3-dev \
    python3-pip \
    python3-wheel \
    python3-setuptools &amp;&amp; \
    rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*

RUN pip3 install --no-cache-dir -U install setuptools pip
RUN pip3 install --no-cache-dir cupy_cuda102==8.0.0rc1 scipy optuna

COPY requirements.txt start.sh run.py uwsgi.ini utils.py /opt/app/
COPY shading_characteristics /opt/app/shading_characteristics

WORKDIR /opt/app
RUN pip install -r requirements.txt
RUN pip install --upgrade 'sentry-sdk[flask]'
RUN pip install uwsgi -I --no-cache-dir

EXPOSE 5000

## Start the server, giving permissions for script
# COPY nginx.conf /etc/nginx
RUN chmod +x ./start.sh 
RUN chmod -R 777 /root
CMD [&quot;./start.sh&quot;] 
</code></pre>

shaharyar
<p><strong>Edit (May 2021)</strong></p>
<p>GKE now officially supports NVIDIA driver version <code>450.102.04</code>, which support <code>CUDA 10.2</code>.<br />
Please note that GKE 1.19.8-gke.1200 and higher is required.</p>
<hr />
<p>As you can see in Nvidia's <a href="https://docs.nvidia.com/deploy/cuda-compatibility/index.html#binary-compatibility__table-toolkit-driver" rel="nofollow noreferrer">website</a>, <code>CUDA 10.2</code> requires Nvidia driver version &gt;= 440.33.</p>
<p>Since the latest Nvidia driver available officially <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/gpus#ubuntu" rel="nofollow noreferrer">in GKE</a> is <code>418.74</code>, the newest <code>CUDA</code> version you can use is <code>10.1</code> at the moment.</p>
<p>If your application, or other dependencies such as PyTorch, can function properly with <code>CUDA 10.1</code>, the fastest solution will be to downgrade your base Docker image with <code>CUDA 10.1</code>.</p>
<p>There are <a href="https://github.com/GoogleCloudPlatform/container-engine-accelerators/issues/119" rel="nofollow noreferrer">unofficial</a> ways to install newer Nvidia Driver versions on GKE nodes running COS, but if it's not a must for you - I'd stick to the official and supported GKE method and use 10.1.</p>
