<p>I want to run an initialisation script on each node, and I want to run them only once.</p>
<p>Here I have the yaml to do some basic initialisation on each node, but once the initialisation script finish execution, the pod exits with <code>exit code: 0</code> and the daemonset restarts the pod, running the initialisation script again and again.</p>
<pre><code>---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: test-init-node-cr
rules:
- apiGroups:
  - &quot;&quot;
  resources:
  - nodes
  verbs:
  - get
  - patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: test-init-node-sa
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: test-init-node-cr
subjects:
- kind: ServiceAccount
  name: test-init-node-sa
  namespace: default
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: test-init-node-sa
  namespace: default
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: test-init-node
  namespace: default
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: test-init-node
      app.kubernetes.io/component: configurator
  # replicas: 3
  template:
    metadata:
      name: test-init-node
      labels:
        app.kubernetes.io/name: test-init-node
        app.kubernetes.io/component: configurator
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: k8s.amazee.io/node-configured
                operator: DoesNotExist
      hostPID: true
      hostNetwork: true
      tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/master
      serviceAccount: test-init-node-sa
      containers:
      - name: init
        env:
        - name: MY_NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        command: 
        - nsenter
        - --mount=/proc/1/ns/mnt
        - --
        - bash
        - -xc
        - |
          echo &quot;starting the magic&quot;
          echo &quot;*   hard  core    unlimited&quot; &gt;&gt;  /etc/security/limits.d/game.conf 
          echo &quot;*   soft  core    unlimited&quot; &gt;&gt;  /etc/security/limits.d/game.conf 

        image: alpine/k8s:1.28.0
        resources:
          requests:
            cpu: 50m
            memory: 50M
        securityContext:
          runAsUser: 0
          privileged: true
</code></pre>
<p>Is there any way for me to prevent the daemonset from restarting the pod if the pod exits?
i.e. ensuring that the initialisation only happen once per node.</p>
<p>I tried adding a preStop but it does not seem to have any effect.
The idea is that if <code>k8s.amazee.io/node-configured</code> is set then the daemonset will not schedule onto that node.</p>
<pre><code>          preStop:
            exec:
              command:
              - /bin/sh&quot;
              - -c
              - kubectl label node &quot;$MY_NODE_NAME&quot; k8s.amazee.io/node-configured=$(date +%s)
</code></pre>
<p>neither does adding a semicolon (well this is expected but I thought why not give it a try)</p>
<pre><code>        command: 
        - nsenter
        - --mount=/proc/1/ns/mnt
        - --
        - bash
        - -xc
        - |
          echo &quot;starting the magic&quot;
          echo &quot;*   hard  core    unlimited&quot; &gt;&gt;  /etc/security/limits.d/game.conf 
          echo &quot;*   soft  core    unlimited&quot; &gt;&gt;  /etc/security/limits.d/game.conf 
        - ; 
        - /bin/sh&quot;
        - -c
        - kubectl label node &quot;$MY_NODE_NAME&quot; k8s.amazee.io/node-configured=$(date +%s)
</code></pre>
<p>Is there any way for me to prevent the daemonset from restarting the pod if the pod exits properly? i.e. ensuring that the initialisation only happen once per node.</p>

Y.H.
<p><em>Disclaimer: I had the same issue few months ago, I don't know if this is the best solution, but at least it works. I've made some research though, and couldn't find a built-in solution, so I ended up with this.</em></p>
<p><em>EDIT: it looks like even GCP resort to this solution: <a href="https://github.com/GoogleCloudPlatform/solutions-gke-init-daemonsets-tutorial." rel="nofollow noreferrer">https://github.com/GoogleCloudPlatform/solutions-gke-init-daemonsets-tutorial.</a></em></p>
<p>The solution is to:</p>
<ol>
<li>make the script inside the daemonset know when the script has already been executed. So that if it restarts, you can just skip the steps</li>
<li>make the script sleep for infinity after executing. This ensure that the daemonset will not restart (unless it's killed, but 1st point will avoid the steps being re-executed)</li>
</ol>
<p>In your specific case, you can transform your script:</p>
<pre class="lang-bash prettyprint-override"><code>echo &quot;starting the magic&quot;
echo &quot;*   hard  core    unlimited&quot; &gt;&gt; /etc/security/limits.d/game.conf
echo &quot;*   soft  core    unlimited&quot; &gt;&gt; /etc/security/limits.d/game.conf
</code></pre>
<p>to:</p>
<pre class="lang-bash prettyprint-override"><code>if [ ! -f /etc/game-conf-limits-updated ]
then
    echo &quot;starting the magic&quot;
    echo &quot;*   hard  core    unlimited&quot; &gt;&gt; /etc/security/limits.d/game.conf
    echo &quot;*   soft  core    unlimited&quot; &gt;&gt; /etc/security/limits.d/game.conf

    touch /etc/game-conf-limits-updated
fi

sleep infinity
</code></pre>
<p>The <code>/etc/game-conf-limits-updated</code> will exist only if the script has been executed once. As I said above, the <code>sleep infinity</code> will prevent the daemonset to restart the pod. But if it restarts for some reason, checking if <code>/etc/game-conf-limits-updated</code> exists (<code>[ ! -f /etc/game-conf-limits-updated ]</code>) will avoid writing again in <code>/etc/security/limits.d/game.conf</code>. And the script will sleep for infinity, avoiding a restart.</p>
<p>Instead of checking if <code>/etc/game-conf-limits-updated</code> exists (and writing a file on your node filesystem), you can also check if the label is set on your node. This is maybe more elegant.</p>
<p>Note that in <a href="https://github.com/GoogleCloudPlatform/solutions-gke-init-daemonsets-tutorial" rel="nofollow noreferrer">GCP tutorial</a>, they use a pod with an init container and a <code>pause</code> container. This is equivalent to my solution with a single container.</p>
<p>The only downside of this is your daemonset pod will run forever on your node.</p>
<p>A better solution would be to configure your node directly (e.g. using custom images for your cluster nodes with already updated files as you wish), but I don't know if it's possible with GKE.</p>
