<p>How to run data science model's docker image on Azure Kubernetes Services (GPU enabled) so that it can utilize the GPU capability of the Kubernetes cluster. The packages which we are using to build models are tensorflow, keras, scikit-learn, etc. Do we need to include Cuda installation steps on Dockerfile?</p>

<p><a href="https://i.stack.imgur.com/jAFp5.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/jAFp5.png" alt="enter image description here"></a></p>

<p>Also find the error below:</p>

<p><a href="https://i.stack.imgur.com/oLAYB.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/oLAYB.png" alt="enter image description here"></a></p>

Chaitanya Kirty
<ol>
<li>create aks with vm size that supports gpu</li>
<li>install nvidia drivers</li>
<li>run your workload</li>
</ol>

<p>for point 2 use the following yaml:</p>

<pre><code>apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: nvidia-device-plugin-daemonset
  namespace: kube-system
spec:
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      # Mark this pod as a critical add-on; when enabled, the critical add-on scheduler
      # reserves resources for critical add-on pods so that they can be rescheduled after
      # a failure.  This annotation works in tandem with the toleration below.
      annotations:
        scheduler.alpha.kubernetes.io/critical-pod: ""
      labels:
        name: nvidia-device-plugin-ds
    spec:
      tolerations:
      # Allow this pod to be rescheduled while the node is in "critical add-ons only" mode.
      # This, along with the annotation above marks this pod as a critical add-on.
      - key: CriticalAddonsOnly
        operator: Exists
      - key: nvidia.com/gpu
        operator: Exists
        effect: NoSchedule
      containers:
      - image: nvidia/k8s-device-plugin:1.11
        name: nvidia-device-plugin-ctr
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop: ["ALL"]
        volumeMounts:
          - name: device-plugin
            mountPath: /var/lib/kubelet/device-plugins
      volumes:
        - name: device-plugin
          hostPath:
            path: /var/lib/kubelet/device-plugins
</code></pre>

<p>official docs on this matter. you can also use <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/gpus" rel="nofollow noreferrer">these official k8s</a> docs on using GPU. I think you should just use a base image that contains GPU drivers, for example you can use the example image MS provides as a base, or something like <code>tensorflow/tensorflow:latest-gpu</code>.</p>
