<p>I'm using Kubernetes(K8) v1.7 and wondering if I can share a GPU among multiple pods. I have an MNIST machine learning program in TensorFlow where GPU Allocation is set to 30% and GPU Growth <code>allow_growth</code> is false. Two manifest files are being used to deploy two separate jobs under K8. When I run them together, one gets scheduled and other waits as the GPU resource is not available. In my manifest file, I set <code>alpha.kubernetes.io/nvidia-gpu</code> to 0.5. When I check GPU processes, it always shows one process is running. If I stop one job from Kubernetes then other gets scheduled and shows up in the GPU processes. FYI, the machine I'm using has only one GPU. So my question is whether the Kubernetes supports GPU sharing or not? Can I share a GPU and define it in the manifest file?</p>

<p>A portion of the Manifest (both jobs have same GPU request)</p>

<pre><code>  resources:
    limits:
      alpha.kubernetes.io/nvidia-gpu: 0.5
</code></pre>

<p>The output of the <code>nvidia-smi</code> command shows 1 process at a time</p>

<pre><code>+-----------------------------------------------------------------------------+
| NVIDIA-SMI 396.26                 Driver Version: 396.26                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 108...  Off  | 00000000:0B:00.0 Off |                  N/A |
|  0%   33C    P2    59W / 275W |    177MiB / 11178MiB |      8%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0     15513      C   python3                                      167MiB |
+-----------------------------------------------------------------------------+ 
</code></pre>

<p>GPU sharing on multiple containers is not supported at the moment, and it is unlikely to be supported anytime soon. You would need to have each virtual machine be a separate Kubernetes node, each with a separate GPU.</p>
