<p>How do I automatically restart Kubernetes pods associated with Daemonsets when their configmap is updated?</p>

<p>As per the <a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#mounted-configmaps-are-updated-automatically" rel="nofollow noreferrer">kubernetes</a> documentation when the configmap volume mount is updated, it automatically updates the pods. However I do not see that happening for Daemonsets. What am I missing?</p>

<p>The below is my configmap</p>

<pre><code>apiVersion: v1
kind: ConfigMap
metadata:
  name: fluent-bit-update
  namespace: default
  labels:
    k8s-app: fluent-bit
data:
  # Configuration files: server, input, filters and output
  # ======================================================
  fluent-bit.conf: |
    [SERVICE]
        Flush         1
        Log_Level     info
        Daemon        off
        Parsers_File  parsers.conf


    @INCLUDE input-kubernetes.conf
    @INCLUDE filter-kubernetes.conf
    @INCLUDE output-logaggregator.conf

  input-kubernetes.conf: |
    [INPUT]
        Name              tail
        Tag               kube.*
        Path              /var/log/containers/abc.log
        Parser            docker
        DB                /var/log/tail-containers-state.db
        DB.Sync           Normal
        Mem_Buf_Limit     5MB
        Skip_Long_Lines   On
        Refresh_Interval  10
        Rotate_Wait      60
        Docker_Mode      On

  filter-kubernetes.conf: |

    [FILTER]
        Name                kubernetes
        Match               kube.*
        Kube_URL            https://kubernetes.default.svc:443
        Kube_CA_File        /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        Kube_Token_File     /var/run/secrets/kubernetes.io/serviceaccount/token
        Kube_Tag_Prefix     kube.var.log.conatiners.
        Merge_Log           On
        Keep_Log            Off
        K8S-Logging.Parser  On
        K8S-Logging.Exclude Off
        Labels              On
        Annotations         On

  output-kubernetes.conf: |

    [OUTPUT]
        Name              cloudwatch
        Match             kube.*
        region            us-west-2
        log_group_name    fluent-bit-cloudwatch
        log_stream_prefix from-fluent-bit
        auto_create_group true

  parsers.conf: |

    [PARSER]
        Name         docker
        Format       json
        Time_Key     time
        Time_Format  %Y-%m-%dT%H:%M:%S.%L
        Time_Keep    On
        Decode_Field_As   escaped_utf8    log    do_next
        Decode_Field_As   json       log

    [PARSER]
        Name         docker_default
        Format       json
        Time_Key     time
        Time_Format  %Y-%m-%dT%H:%M:%S.%L
        Time_Keep    On
</code></pre>

<p>&amp; my daemonset manifest file</p>

<pre><code>apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: fluent-bit-update
  namespace: default
  labels:
    k8s-app: fluent-bit-logging
    version: v1
    kubernetes.io/cluster-service: "true"
spec:
  template:
    metadata:
      labels:
        k8s-app: fluent-bit-logging
        version: v1
        kubernetes.io/cluster-service: "true"
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "2020"
        prometheus.io/path: /api/v1/metrics/prometheus
    spec:
      containers:
        - name: aws-for-fluent-bit
          image: amazon/aws-for-fluent-bit:latest
          imagePullPolicy: Always
          ports:
            - containerPort: 2020
          volumeMounts:
            - name: varlog
              mountPath: /var/log
            - name: varlibdockercontainers
              mountPath: /var/lib/docker/containers
              readOnly: true
            - name: fluent-bit-volume
              mountPath: /fluent-bit/etc/
      terminationGracePeriodSeconds: 10
      volumes:
        - name: varlog
          hostPath:
            path: /var/log
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
        - name: fluent-bit-volume
          configMap:
            name: fluent-bit-update
      serviceAccountName: fluent-bit
      tolerations:
        - key: node-role.kubernetes.io/master
          operator: Exists
          effect: NoSchedule
        - operator: "Exists"
          effect: "NoExecute"
        - operator: "Exists"
          effect: "NoSchedule"
</code></pre>

<p>When I updated the <code>Path</code> field in configmap to read another log file,  though I see the volume mounts getting updated, I do not see the pods picking up the change unless I delete and recreate the daemonset. </p>

<p>Is there a way to achieve this automagically without restarting the daemonset? I would appreciate some guidance on this. Thanks</p>

<p>As per the documentation you pointed, Kubernetes will update the configmap (fluent-bit configuration in your case) but it is your application's duty to pick the updated config. Generally, the application takes the config during start-time but to update the config periodically, either your application should support this or another module (like a config updater) should be used which will restart your app (without restarting pod) when there is a change in config.
For fluent-bit you can refer this <a href="https://github.com/fluent/fluent-bit/issues/365" rel="noreferrer">github issue</a> for dynamic configuration.
Prometheus already supports it like <a href="https://github.com/coreos/prometheus-operator/tree/master/cmd/prometheus-config-reloader" rel="noreferrer">this</a>.</p>
