<p>I have an application designed to run as a K8s application, and it imports some dependencies (that I don't own) that run <code>exec.Cmd</code>s. This is fine, except I want to capture those logs. For some reason, when I do:</p>
<pre><code>r := bufio.NewReader(os.Stdout)

...

line, err := r.ReadString('\n')
</code></pre>
<p>An error is thrown saying that <code>/dev/stdout</code> is a <code>bad file descriptor</code>. How can this be? Isn't that the standard local destination for console output?</p>
<p><code>kubectl logs</code> seems to be able to capture the output, and more specifically, our central log forwarder is able to capture it as well. But trying to capture logs from the kube API server inside the container that's actually generating those logs seems kinda silly... Is there a better way to do this?</p>

jayjyli
<p>Generally, <code>stdin</code> is a read-only stream for retrieving input written to your program, while <code>stdout</code> is a write-only stream for sending output written by your program. <em>In other words, nobody can read from /dev/stdout, except Chuck Norris.</em></p>
<p>By default, <code>stdout</code> is &quot;pointing&quot; to your terminal. But it is possible to redirect <code>stdout</code> from your terminal to a file. This redirection is set up before your program is started.</p>
<p>What usually happens, is the following: The container runtime redirects <code>stdout</code> of the process of your container to a file on the node where your container is running (e.g., <code>/var/log/containers/&lt;container-name&gt;-&lt;container-id&gt;.log</code>). When you request logs with <code>kubectl logs</code>, kubectl connects to kube-apiserver, which connects to the kubelet on the node running your container and asks it to send back the content from the log file.</p>
<p>Also take a look at <a href="https://kubernetes.io/docs/concepts/cluster-administration/logging/" rel="nofollow noreferrer">https://kubernetes.io/docs/concepts/cluster-administration/logging/</a> which explains the various logging design approaches.</p>
<hr />
<p>A solution, which from a security and portability perspective you would definitely NOT implement, is to add a <code>hostPath</code> mount in your container mounting the <code>/var/log/containers</code> directory of your node and to access the container log directly.</p>
<hr />
<p>A proper solution might be to change the command of your image and to write output to <code>stdout</code> of your container and also to a local file within your container. This can be achieved using the <code>tee</code> command. Your application can then read back the log from this file. But keep in mind, that without proper rotation, the log file will grow until your container is terminated.</p>
<pre><code>apiVersion: v1
kind: Pod
metadata:
  name: log-to-stdout-and-file
spec:
  containers:
  - image: bash:latest
    name: log-to-stdout-and-file
    command: 
    - bash
    - -c
    - '(while true; do date; sleep 10; done) | tee /tmp/test.log'
</code></pre>
<hr />
<p>A little more complex solution would be, to replace the log file in the container with a named pipe file created with <code>mkfifo</code>. This avoids the growing file size problem (as long as your application is continuously reading the log from the named pipe file).</p>
<pre><code>apiVersion: v1
kind: Pod
metadata:
  name: log-to-stdout-and-file
spec:
  # the init container creates the fifo in an empty dir mount
  initContainers:
  - image: bash:latest
    name: create-fifo
    command: 
    - bash
    - -c
    - mkfifo /var/log/myapp/log
    volumeMounts:
    - name: ed
      mountPath: /var/log/myapp

  # the actual app uses tee to write the log to stdout and to the fifo
  containers:
  - image: bash:latest
    name: log-to-stdout-and-fifo
    command: 
    - bash
    - -c
    - '(while true; do date; sleep 10; done) | tee /var/log/myapp/log'
    volumeMounts:
    - name: ed
      mountPath: /var/log/myapp

  # this sidecar container is only for testing purposes, it reads the
  # content written to the fifo (this is usually done by the app itself)
  #- image: bash:latest
  #  name: log-reader
  #  command: 
  #  - bash
  #  - -c
  #  - cat /var/log/myapp/log
  #  volumeMounts:
  #  - name: ed
  #    mountPath: /var/log/myapp

  volumes:
  - name: ed
    emptyDir: {}
</code></pre>
