<p>Today when I using <code>duc</code> command to check the kubernetes(v1.21) cluster host disk usage like this:</p>
<pre><code>[root@k8smasterone log]# duc ls -Fg /var/log/
  5.4G messages-20220515             [+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++]
  3.9G messages                      [++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++                                                   ]
571.1M messages-20220508             [++++++++++++++++++
</code></pre>
<p>the kubelet output too large files, what should I do to limit the kubelet log size? I have read this issue to talk about the log should handle by kubelet: <a href="https://github.com/containerd/containerd/issues/4830" rel="nofollow noreferrer">https://github.com/containerd/containerd/issues/4830</a>. but I did not found any possible solution to solve the kubelet log rotate problem. PS: I am using containerd not docker right now. This is my journald config:</p>
<pre><code>[root@k8smasterone log]# cat /etc/systemd/journald.conf
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.
#
# Entries in this file show the compile time defaults.
# You can change settings by editing this file.
# Defaults can be restored by simply deleting this file.
#
# See journald.conf(5) for details.

[Journal]
#Storage=auto
Compress=yes
#Seal=yes
#SplitMode=uid
#SyncIntervalSec=5m
#RateLimitInterval=30s
#RateLimitBurst=1000
SystemMaxUse=50M
#SystemKeepFree=
SystemMaxFileSize=20M
#RuntimeMaxUse=
#RuntimeKeepFree=
#RuntimeMaxFileSize=
MaxRetentionSec=1week
#MaxFileSec=1month
#ForwardToSyslog=yes
#ForwardToKMsg=no
#ForwardToConsole=no
#ForwardToWall=yes
#TTYPath=/dev/console
#MaxLevelStore=debug
#MaxLevelSyslog=debug
#MaxLevelKMsg=notice
#MaxLevelConsole=info
#MaxLevelWall=emerg
#LineMax=48K
</code></pre>

<p>You can tweak the following two parameters in the <code>kubelet</code> <a href="https://v1-21.docs.kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/" rel="nofollow noreferrer">config</a>:</p>
<blockquote>
<p><strong><code>containerLogMaxSize</code></strong> string  A quantity defines the maximum size of the
container log file before it is rotated. For example &quot;5Mi&quot; or
&quot;256Ki&quot;. Dynamic Kubelet Config (beta): If dynamically updating this
field, consider that it may trigger log rotation. Default: &quot;10Mi&quot;</p>
<p><strong><code>containerLogMaxFiles</code></strong> int32  Maximum number of container log files that
can be present for a container. Dynamic Kubelet Config (beta): If
dynamically updating this field, consider that lowering it may cause
log files to be deleted. Default: 5</p>
</blockquote>
<p>Example:</p>
<pre><code>sudo vi /etc/kubernetes/kubelet-config.yaml
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
nodeStatusUpdateFrequency: &quot;10s&quot;
failSwapOn: True
authentication:
  anonymous:
    enabled: false
  webhook:
    enabled: True
  x509:
    clientCAFile: /etc/kubernetes/ssl/ca.crt
authorization:
  mode: Webhook
staticPodPath: /etc/kubernetes/manifests
cgroupDriver: systemd
containerLogMaxFiles: 5    # Maximum number of container logs to retain. 
containerLogMaxSize: 1Mi   # Change the size of /var/log/containers/&lt;pod-name&gt;/log files size to 1M max.
maxPods: 110
address: 192.168.22.5
readOnlyPort: 0
healthzPort: 10248
healthzBindAddress: 127.0.0.1
kubeletCgroups: /systemd/system.slice
clusterDomain: cluster.local
protectKernelDefaults: true
rotateCertificates: true
clusterDNS:
- 8.8.8.8
kubeReserved:
  cpu: 200m
  memory: 512Mi
resolvConf: &quot;/run/systemd/resolve/resolv.conf&quot;
eventRecordQPS: 5
shutdownGracePeriod: 60s
shutdownGracePeriodCriticalPods: 20s
</code></pre>
<p>Reload the kubelet config:</p>
<pre><code>sudo systemctl daemon-reload &amp;&amp; sudo systemctl restart kubelet
</code></pre>
