<p><a href="https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/" rel="nofollow noreferrer">Official documentation</a> on enabling the GPU support states:</p>

<blockquote>
  <p>A special alpha feature gate Accelerators has to be set to true across
  the system: --feature-gates="Accelerators=true".</p>
</blockquote>

<p>I am having trouble decoding the "set to true across the system" part.</p>

<p>I have discovered that <a href="https://kubernetes.io/docs/admin/kubelet/" rel="nofollow noreferrer">kubelet</a>, <a href="https://kubernetes.io/docs/admin/kube-apiserver/" rel="nofollow noreferrer">kube-apiserver</a>, and <a href="https://kubernetes.io/docs/admin/kube-controller-manager/" rel="nofollow noreferrer">kube-controller-manager</a> all have the --feature-gates runtime parameter. The specification states that they all listen on modifications to config file.</p>

<p>Any help with where those config files are how I can enable the --feature-gates="Accelerators=true" option in them?</p>

<p>I did try adding the option to /etc/kubernetes/manifests/kube-apiserver.yaml:
spec:</p>

<pre><code>  containers:
  - command:
    - kube-apiserver
    - -- &lt;...&gt;
    - --feature-gates=Accelerators=true
</code></pre>

<p>However, that causes kube-apiserver to stop and never come back.</p>

<p>In the end I found the following workaround <a href="https://github.com/kubernetes/kops/pull/2257/files" rel="nofollow noreferrer">here</a>:</p>

<blockquote>
  <p>3.I Add GPU support to the Kubeadm configuration, while cluster is not initialized. This has to be done for every node across your cluster,
  even if some of them don't have any GPUs.</p>
  
  <p>sudo vim /etc/systemd/system/kubelet.service.d/&lt;>-kubeadm.conf
  Therefore, append ExecStart with the flag
  --feature-gates="Accelerators=true", so it will look like this:</p>
  
  <p>ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS [...]
  --feature-gates="Accelerators=true"
  3.II Restart kubelet</p>
  
  <p>sudo systemctl daemon-reload sudo systemctl restart kubelet</p>
</blockquote>

<p>However, I believe that the above approach is not how Kubernetes developers intended for this feature to be enabled. Any help would be appreciated.</p>

<hr>

<p>[Edit] I was able to turn on the option on both api-server and controller-manager - neither gave the desired result of gpu becoming visible.</p>

<p>So it's the kubelet service that needs to get this option.</p>

<p>The question becomes: how can the option be set via the kubelet config file?</p>

<p>I use Ubuntu16.04.</p>

<p><code>Add --feature-gates="Accelerators=true"</code> to <code>KUBELET_ARGS</code> in file <code>/etc/kubernetes/kubelet</code> should be fine.</p>
