<p>I have a docker image of my kafka_consumer code which is consuming messages from a topic. Now what i did was that i created a pod out of my docker image which is running successfully. When i issue the command
kubectl logs  it just prints first three lines of logs and then exits. When i run the docker image it gives me complete output where in consumer record is printed several times. What is wrong with kubernetes logs then?</p>

<p>I tried using <code>kubectl logs &lt;pod-name&gt;</code> which returns me only 3 lines of logs.</p>

<p>I expect the output to show the proper detailed messages which will look like this:</p>

<pre><code>log4j:WARN No appenders could be found for logger (org.apache.kafka.clients.consumer.ConsumerConfig).

log4j:WARN Please initialize the log4j system properly.

log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

ConsumerRecord(topic = tt, partition = 0, offset = 399219, CreateTime = 1565941033699, serialized key size = 3, serialized value size = 6, headers = RecordHeaders(headers = [], isReadOnly = false), key = key, value = hello )
</code></pre>

<p>When i use kubectl i get only first three lines of log4j and not the ConsumerRecord.</p>

Vaibhav Srivastava
<p>When you <code>kubectl logs</code> to any pod, you're <a href="https://kubernetes.io/docs/concepts/cluster-administration/logging/#logging-at-the-node-level" rel="nofollow noreferrer">querying the <code>stdout/stderr</code> streams sent to the node's log files</a> (<code>/var/log/*</code>), which in turn depends on the underlying host OS logrotate.</p>

<p>You can try to determine if this is the problem by either, comparing the logs from <code>kubectl logs</code> versus the ones sent into a <a href="https://kubernetes.io/docs/concepts/cluster-administration/logging/#cluster-level-logging-architectures" rel="nofollow noreferrer">logging backend</a> (if any, for instance <em>Fluentd</em> or the <em>ELK stack</em>) or directly <em>SSHing</em> into the node and locating them directly in the logging path.</p>

<p>There aren't many details on how are you running your Kubernetes cluster, but the information provided strongly suggests that this is a node level issue, specific to how pod logs are being managed internally.</p>

<p>Finally, the approach using <code>kubectl exec &lt;podname&gt; -- /bin/bash</code> is a good idea to determine if the container is locally storing these logs and the issue is when they're sent into the host node.</p>
