<p>I am using the ELK stack (elasticsearch, logsash, kibana) for log processing and analysis in a Kubernetes (minikube) environment.  To capture logs I am using filebeat.  Logs are propagated successfully from filebeat through to elasticsearch and are viewable in Kibana.  </p>

<p>My problem is that I am unable to get the pod name of the actual pod issuing log records.  Rather I only get the filebeat podname which is gathering log files and not name of the pod that is originating log records.</p>

<p>The information I can get from filebeat are (as viewed in Kibana)</p>

<ul>
<li>beat.hostname: the value of this field is the filebeat pod name</li>
<li>beat.name: value is the filebeat pod name</li>
<li>host: value is the filebeat pod name</li>
</ul>

<p>I can also see/discern container information in Kibana which flow through from filebeat / logstash / elasticsearch:</p>

<ul>
<li>app: value is {log-container-id}-json.log</li>
<li>source: value is /hostfs/var/lib/docker/containers/{log-container-id}-json.log</li>
</ul>

<p>As shown above, I seem to be able to get the container Id but not the pod name.</p>

<p>To mitigate the situation, I could probably embed the pod-name in the actual log message and parse it from there, but I am hoping there is a solution in which I can configure filebeat to emit actual pod names.</p>

<p>Does anyone now how to configure filebeat (or other components) to capture kubernetes (minikube) pod names in their logs?</p>

<p>My current filebeat configuration is listed below:</p>

<p>ConfigMap is shown below:</p>

<pre><code>apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat
  namespace: logging
  labels:
    component: filebeat
data:
  filebeat.yml: |
    filebeat.prospectors:

    - input_type: log
      tags:
      - host
      paths:
      - "/hostfs/var/log"
      - "/hostfs/var/log/*"
      - "/hostfs/var/log/*/*"
      exclude_files:
      - '\.[0-9]$'
      - '\.[0-9]\.gz$'

    - input_type: log
      tags:
      - docker
      paths:
      - /hostfs/var/lib/docker/containers/*/*-json.log
      json:
        keys_under_root: false
        message_key: log
        add_error_key: true
      multiline:
        pattern: '^[[:space:]]+|^Caused by:'
        negate: false
        match: after

    output.logstash:
      hosts: ["logstash:5044"]

    logging.level: info
</code></pre>

<p>DamemonSet is shown below:</p>

<pre><code>apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: filebeat
  namespace: logging
spec:
  template:
    metadata:
      labels:
        component: filebeat
    spec:
      containers:
      - name: filebeat
        image: giantswarm/filebeat:5.2.2
        imagePullPolicy: IfNotPresent
        resources:
          limits:
            cpu: 100m
          requests:
            cpu: 100m
        volumeMounts:
        - name: config
          mountPath: /etc/filebeat
          readOnly: true
        - name: hostfs-var-lib-docker-containers
          mountPath: /hostfs/var/lib/docker/containers
          readOnly: true
        - name: hostfs-var-log
          mountPath: /hostfs/var/log
          readOnly: true
      volumes:
      - name: config
        configMap:
          name: filebeat
      - name: hostfs-var-log
        hostPath:
          path: /var/log
      - name: hostfs-var-lib-docker-containers
        hostPath:
      path: /var/lib/docker/containers
</code></pre>

<p>I have achieved what you looking for, by assigning a group of specific pods to a namespace, and now can query the log I look for using a combination of namespace, pod name and container name which is also included in generated log which is piped by file beat without any extra effort as you can see here
<img src="https://i.stack.imgur.com/230U3.png" alt="image"> </p>
