<p>I've been reading about how to setup audit in kubernetes <a href="https://kubernetes.io/docs/tasks/debug-application-cluster/audit/#audit-policy" rel="nofollow noreferrer">here</a> which basically says that in order to enable audit I have to specify a yaml policy file to kube-apiserver when starting it up, by using the flag <code>--audit-policy-file</code>.</p>

<p>Now, there are two things I don't understand about how to achieve this:</p>

<ol>
<li>What's the proper way to add/update a startup parameter of the command that runs kube-apiserver? I cannot update the pod, so do I need to clone the pod somehow? Or should I use <code>kops edit cluster</code> as suggested here: <a href="https://github.com/kubernetes/kops/blob/master/docs/cluster_spec.md#kubeapiserver" rel="nofollow noreferrer">https://github.com/kubernetes/kops/blob/master/docs/cluster_spec.md#kubeapiserver</a>. Surprisingly, kubernetes does not create a deployment for this, should I create it myself?</li>
<li>In particular to setup audit I have to pass a yaml file as a startup argument. How do I upload/make available this yaml file in order to make a <code>--audit-policy-file=/some/path/my-audit-file.yaml</code>. Do I create a configMap with it and/or a volume? How can I reference this file afterwards, so it's available in the filesystem when kube-apiserver startup command runs?</li>
</ol>

<p>Thanks!</p>

<blockquote>
  <p>What's the proper way to add/update a startup parameter of the command that runs kube-apiserver?</p>
</blockquote>

<p>In 99% of the ways that I have seen kubernetes clusters deployed, the <code>kubelet</code> binary on the Nodes reads the kubernetes descriptors in <code>/etc/kubernetes/manifests</code> on the host filesystem and runs the Pods described therein. So, the answer to the first question is to edit -- or cause the configuration management tool you are using to update -- the file <code>/etc/kubernetes/manifests/kube-apiserver.yaml</code> (or hopefully a very similarly named file). If you have multiple master Nodes, you will need to repeat that process for all master Nodes. In <em>most</em> cases, the <code>kubelet</code> binary will see the change to the manifest file and will restart the apiserver's Pod automatically, but in the worst case restarting <code>kubelet</code> may be required.</p>

<p>Be sure to watch the output of the newly started apiserver's docker container to check for errors, and only roll that change out to the other apiserver manifest files after you have confirmed it works correctly.</p>

<blockquote>
  <p>How can I reference this file afterwards, so it's available in the filesystem when kube-apiserver startup command runs?</p>
</blockquote>

<p>Roughly the same answer: either via ssh or any on-machine configuration management tool. The only asterisk to this one is that since the apisever's manifest file is a normal <code>Pod</code> declaration, one will wish to be mindful of the <code>volume:</code>s and <code>volumeMount:</code>s just like you would for any other in-cluster <code>Pod</code>. That is likely to be fine if your <code>audit-policy.yaml</code> lives in or under <code>/etc/kubernetes</code>, since that directory is already volume mounted into the Pod (again: most of the time). It's writing out the audit log file that will most likely require changes, since unlike the rest of the config the log file path cannot be <code>readOnly: true</code> and thus will at minimum require a 2nd <code>volumeMount</code> without the <code>readOnly: true</code>, and likely will require a 2nd <code>volume: hostPath:</code> to make the log directory visible into the Pod.</p>

<p>I actually haven't tried using a <code>ConfigMap</code> for the apiserver itself, as that's very meta. But, in a multi-master setup, I don't know that it's <em>impossible</em>, either. Just be cautious, because in such a self-referential setup it would be very easy to bring down all masters with a bad configuration since they wouldn't be able to communicate with themselves to read the updated config.</p>
