<p>I have defined a K8S configuration which deploy a metricbeat container. Below is the configuration file. But I got an error when run <code>kubectl describe pod</code>:</p>
<pre><code>Events:
  Type     Reason       Age                   From               Message
  ----     ------       ----                  ----               -------
  Normal   Scheduled    5m24s                 default-scheduler  Successfully assigned default/metricbeat-6467cc777b-jrx9s to ip-192-168-44-226.ap-southeast-2.compute.internal
  Warning  FailedMount  3m21s                 kubelet            Unable to attach or mount volumes: unmounted volumes=[config], unattached volumes=[default-token-4dxhl config modules]: timed out waiting for the condition
  Warning  FailedMount  74s (x10 over 5m24s)  kubelet            MountVolume.SetUp failed for volume &quot;config&quot; : configmap &quot;metricbeat-daemonset-config&quot; not found
  Warning  FailedMount  67s                   kubelet            Unable to attach or mount volumes: unmounted volumes=[config], unattached volumes=[config modules default-token-4dxhl]: timed out waiting for the condition
</code></pre>
<p>based on the error message, it says <code>configmap &quot;metricbeat-daemonset-config&quot; not found</code> but it does exist in below configuration file. why does it report this error?</p>
<pre><code>apiVersion: v1
kind: ConfigMap
metadata:
  name: metricbeat-daemonset-config
  namespace: kube-system
  labels:
    k8s-app: metricbeat
data:
  metricbeat.yml: |-
    metricbeat.config.modules:
      # Mounted `metricbeat-daemonset-modules` configmap:
      path: ${path.config}/modules.d/*.yml
      # Reload module configs as they change:
      reload.enabled: false

    processors:
      - add_cloud_metadata:

    output.elasticsearch:
      hosts: ['${ELASTICSEARCH_HOST}:${ELASTICSEARCH_PORT:9200}']
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: metricbeat-daemonset-modules
  labels:
    k8s-app: metricbeat
data:
  aws.yml: |-
    - module: aws
      access_key_id: 'xxxx'
      secret_access_key: 'xxxx'
      period: 600s
      regions:
        - ap-southeast-2
      metricsets:
        - elb
        - natgateway
        - rds
        - transitgateway
        - usage
        - vpn
        - cloudwatch
      metrics:
        - namespace: &quot;*&quot;
---
# Deploy a Metricbeat instance per node for node metrics retrieval
apiVersion: apps/v1
kind: Deployment
metadata:
  name: metricbeat
  labels:
    k8s-app: metricbeat
spec:
  selector:
    matchLabels:
      k8s-app: metricbeat
  template:
    metadata:
      labels:
        k8s-app: metricbeat
    spec:
      terminationGracePeriodSeconds: 30
      hostNetwork: true
      containers:
      - name: metricbeat
        image: elastic/metricbeat:7.11.1
        env:
        - name: ELASTICSEARCH_HOST
          value: es-entrypoint
        - name: ELASTICSEARCH_PORT
          value: '9200'
        resources:
          limits:
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 100Mi
        volumeMounts:
        - name: config
          mountPath: /etc/metricbeat.yml
          readOnly: true
          subPath: metricbeat.yml
        - name: modules
          mountPath: /usr/share/metricbeat/modules.d
          readOnly: true
      volumes:
        - name: config
          configMap:
            defaultMode: 0640
            name: metricbeat-daemonset-config
        - name: modules
          configMap:
            defaultMode: 0640
            name: metricbeat-daemonset-modules

</code></pre>

Joey Yi Zhao
<p>There's a good chance that the other resources are ending up in namespace <code>default</code> because they do not specify a namespace, but the config does (<code>kube-system</code>). You should probably put all of this in its own <code>metricbeat</code> namespace.</p>
