<p>I am really new to kubernetes and have testing app with redis and mongodb running in GCE. I would like to grap my log files with fluentd and send them to logz:</p>

<p>I use the following fluentd config file. I tested a similar version on my local machine.</p>

<pre><code>&lt;source&gt;
    @type tail
    path /var/log/containers/squidex*.log
    pos_file /var/log/squidex.log.pos
    tag squidex.logs
    format json
&lt;/source&gt;

&lt;match squidex.logs&gt;
    @type copy
    &lt;store&gt;
        @type logzio_buffered
        endpoint_url https://listener.logz.io:8071?token=...
        output_include_time true
        output_include_tags true
        buffer_type file
        buffer_path /fluentd/log/squidex.log.buffer
        flush_interval 10s
        buffer_chunk_limit 1m
    &lt;/store&gt;
    &lt;store&gt;
        @type stdout
    &lt;/store&gt;
&lt;/match&gt;
</code></pre>

<p>My kubernetes configuration is:</p>

<pre><code>---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: fluentd-logging
  labels:
    app: fluentd-logging
spec:
  template:
    metadata:
      labels:
        app: fluentd-logging
    spec:
      containers:
      - name: fluentd
        image: gcr.io/squidex-157415/squidex-fluentd:latest
        resources:
          limits:
            memory: 200Mi
          requests:
            cpu: 40m
        volumeMounts:
        - name: varlog
          mountPath: /var/log
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
</code></pre>

<p>Almost everything works, but when I run the fluentd pods I see the following entries in the log output from these pods:</p>

<pre><code>2017-04-22T09:49:22.286740784Z 2017-04-22 09:49:22 +0000 [warn]: 
/var/log/containers/squidex-282724611-3nhtw_default_squidex-ed7c437e677d3438c137cdc80110d106339999a6ba8e495a5752fe6d5da9e70d.log unreadable. 
It is excluded and would be examined next time
</code></pre>

<p>How can I get permissions to those log files?</p>

<p>This is not a permission issue but broken symlinks.
Kubernetes is using symbolic links from <code>/var/log/containers</code> to <code>/var/log/pods</code> to <code>/var/lib/docker/containers</code>. You can confirm this from any node of your cluster using <code>ls -la</code></p>

<p>Your DaemonSet configuration should include something like:</p>

<pre><code>volumeMounts:
- name: varlog
  mountPath: /var/log/
  readOnly: true
  - name: varlibdockercontainers
  mountPath: /var/lib/docker/containers
  readOnly: true
[...]
volumes:
- name: varlog
  hostPath:
    path: /var/log/
- name: varlibdockercontainers
  hostPath:
    path: /var/lib/docker/containers
</code></pre>

<p>This way, you are mounting both the logs files directory and the symlinks of symlinks so your fluentd can read everything.</p>
