<p>When using the image <code>tensorflow/serving:latest-devel-gpu</code> on Kuberenetes, the GPU isn't being used. </p>

<p>I don't do anything fancy with it, simply pass server.conf and model files.</p>

<p>The default runtime is nvidia-docker, and my other GPU pod is able to use the GPU.</p>

<p>The only error in the log:</p>

<blockquote>
  <p>E
  external/org_tensorflow/tensorflow/stream_executor/cuda/cuda_driver.cc:397
  ] failed call to cuInit: CUresult(-1)</p>
</blockquote>

<p>Something else which is interesting:</p>

<blockquote>
  <p>I
  external/org_tensorflow/tensorflow/stream_executor/cuda/cuda_diagnostics.cc:189]
  libcuda reported version is: Not found: was unable to find libcuda.so
  DSO loaded into this program</p>
</blockquote>

aclowkay
<p>update you dockerfile with </p>

<pre><code>RUN rm /usr/local/cuda/lib64/stubs/libcuda.so.1
</code></pre>

<p>or extend the serving devel gpu image from docker hub, with one extra line</p>

<pre><code>FROM tensorflow/serving:1.9.0-devel-gpu
RUN rm /usr/local/cuda/lib64/stubs/libcuda.so.1
</code></pre>
