<p>As a part of debug i need to track down events like pod creation and removal. in my kubernetes set up I am using logging level 5.</p>

<p>Kube api server, scheduler, controller, etcd are running on master node and the minion nodes is running kubelet and docker. </p>

<p>I am using <code>journalctl</code> to get K8s logs on master node as well on worker node. On worker node i can see logs from Docker and Kubelet. These logs contain events as i would expect as i create and destroy pods.</p>

<p>However on Master node i dont see any relevant logs which may indicate a pod creation or removal request handling. </p>

<p><strong>what other logs or method i can use to get such logs from Kubernetes master components  (API server, controller, scheduler, etcd)?</strong></p>

<p>i have checked the logs from API server, controller, scheduler, etcd pods; they dont seem to have such information.</p>

<p>thanks</p>

ankit patel
<p><strong>System component logs:</strong></p>

<p>There are two types of system components: </p>

<ul>
<li><p>those that run in a container </p></li>
<li><p>and those that do not run in a container. </p></li>
</ul>

<p>For example:</p>

<p>The Kubernetes scheduler and kube-proxy run in a container</p>

<p>The kubelet and container runtime, for example Docker, do not run in containers.</p>

<p>On machines with systemd, the kubelet and container runtime write to journald. If systemd is not present, they write to .log files in the /var/log directory. System components inside containers always write to the /var/log directory, bypassing the default logging mechanism. They use the klog logging library.</p>

<p><strong>Master components logs:</strong></p>

<p>Get them from those containers running on master nodes.</p>

<pre><code>$ 
$ docker ps | grep apiserver
d6af65a248f1        af20925d51a3                 "kube-apiserver --ad…"   2 weeks ago         Up 2 weeks                              k8s_kube-apiserver_kube-apiserver-minikube_kube-system_177a3eb80503eddadcdf8ec0423d04b9_0
5f0e6b33a29f        k8s.gcr.io/pause-amd64:3.1   "/pause"                 2 weeks ago         Up 2 weeks                              k8s_POD_kube-apiserver-minikube_kube-system_177a3eb80503eddadcdf8ec0423d04b9_0
$ 
$ 
$ docker logs -f d6a  
</code></pre>

<p>But all of this approach to logging is just for testing , you should stream all the logs , ( app logs , container logs , cluster level logs , everything) to a centeral logging system such as ELK or EFK.</p>
