<p>I have created a pod that produces several millions of log lines.</p>

<p>However <code>kubectl logs -f &lt;pod_name&gt;</code> at some point stops and the output freezes.</p>

<p>Here is my experiment:</p>

<pre><code>kubectl logs --follow=true test-logs-job-13263-t5dbx | tee file1
kubectl logs --follow=true test-logs-job-13263-t5dbx | tee file2
kubectl logs --follow=true test-logs-job-13263-t5dbx | tee file3
</code></pre>

<p>Each of the above processes was interrupted (<code>Ctrl+C</code>) as soon as I stopped receiving any input on my screen.</p>

<p>So:</p>

<pre><code>$ wc -l file*
  106701 file1
  106698 file2
  106698 file3
  320097 total
</code></pre>

<p>It is worth mentioning also that each of these files contains different logs, i.e. the actual logs that were streamed at the time the <code>kubectl logs -f</code> command was actually executed.</p>

<p>Is there a setting that limits the max  amount of logs to be streamed to the above number? (~10700) </p>

<p>My cluster is on GKE for what that matters.</p>

<p><strong>edit</strong>: For some reason this seem to happen <strong>only</strong> on GKE; when I run the same experiment on another k8s cluster (<a href="https://www.katacoda.com/courses/kubernetes/playground" rel="nofollow noreferrer"><code>katakoda</code></a>) the logs were streamed without any issue whatsoever.</p>

<p>Kubernetes does not provide logging itself but uses what the container engine provides. If you were running without kubernetes on a plain docker installation the default configuration of docker is to use the json-file driver and keep 10 mb of logs per container.
See <a href="https://docs.docker.com/config/containers/logging/json-file/" rel="nofollow noreferrer">https://docs.docker.com/config/containers/logging/json-file/</a>
If you need a huge amount of logs consider using central logging infrastructure and configure log forwarding. (This is supported by docker as well for several logging backends, see <a href="https://docs.docker.com/config/containers/logging/configure/#supported-logging-drivers" rel="nofollow noreferrer">https://docs.docker.com/config/containers/logging/configure/#supported-logging-drivers</a> )</p>

<p>There is a bug reported that log rotation can interrupt the streaming of kubectl logs --follow: <a href="https://github.com/kubernetes/kubernetes/issues/28369" rel="nofollow noreferrer">https://github.com/kubernetes/kubernetes/issues/28369</a></p>
