<p>I am trying to understand the interaction between Docker and Fluentd in a K8s cluster. I have seen places where you need to configure Docker to output to a logging driver, and Fluentd can be used as logging driver, like <a href="https://www.fluentd.org/guides/recipes/docker-logging" rel="nofollow noreferrer">here</a>.</p>
<p>On the other hand, I have seen posts (like <a href="https://stackoverflow.com/questions/51097947/fluentd-logging-on-kubernetes-skips-logs-on-log-rotation">this</a> or <a href="https://medium.com/kubernetes-tutorials/cluster-level-logging-in-kubernetes-with-fluentd-e59aa2b6093a" rel="nofollow noreferrer">this</a>) where Docker does not know the existence of Fluentd as a DaemonSet.</p>
<p>My whole intention is to do log rotation, however I am not sure if having Fluentd in place will actually rotate the logs Docker writes on, so I do not end up with the whole storage space in the node taken up by the logs over time. Is it enough to use FluentD DaemonSet without Docker knowing the existence of Fluentd?, o I need to somehow connect Docker to Fluentd with a driver as well?</p>

<p>Per <a href="https://kubernetes.io/docs/concepts/cluster-administration/logging/" rel="nofollow noreferrer">official k8s logging architecture</a> docker (or any other runtime) does not need to know about FluentBit, Fluentd, Filebeat, or any other log collector you use. In fact, you can use multiple log collectors a time!</p>
<p><a href="https://i.stack.imgur.com/kNHZ8.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/kNHZ8.png" alt="enter image description here" /></a></p>
<p>The same document states that k8s is not responsible for log rotation, so you set up a logrotate yourself. Fluentd/FluentBit daemon on the other end also does not rotate log files, but it does able to track log rotation and adjust the tail cursor accordingly (by default).</p>
<p>By far the easiest way to implement the architecture is</p>
<ul>
<li>Leave kubelet &amp; docker settings at default</li>
<li>Ensure the app logs stdout/stderr</li>
<li>Ensure there's logrotate: many k8s worker AMIs, e.g. EKS already have it.</li>
<li>Setup FluentBit log collector daemonset <a href="https://github.com/helm/charts/tree/master/stable/fluent-bit" rel="nofollow noreferrer">https://github.com/helm/charts/tree/master/stable/fluent-bit</a></li>
</ul>
