<p>I initialize a new cluster on the master node with <code>kubeadm init --pod-network-cidr=10.1.0.0/16</code> and install Calico and everything seems to be working:</p>

<pre><code>sysadm@master$ sudo kubectl get pods --all-namespaces -o wide
[sudo] password for sysadm:
NAMESPACE     NAME                                            READY   STATUS    RESTARTS   AGE     IP              NODE                    NOMINATED NODE   READINESS GATES
kube-system   calico-node-ntzn2                               2/2     Running   0          4m9s    192.168.0.249   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-hqmn2                         1/1     Running   0          4m9s    10.1.0.2        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-nfgr5                         1/1     Running   0          4m9s    10.1.0.3        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   etcd-localhost.localdomain                      1/1     Running   0          3m4s    192.168.0.249   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-apiserver-localhost.localdomain            1/1     Running   0          3m18s   192.168.0.249   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-controller-manager-localhost.localdomain   1/1     Running   0          3m23s   192.168.0.249   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-proxy-xgnlb                                1/1     Running   0          4m9s    192.168.0.249   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-scheduler-localhost.localdomain            1/1     Running   0          3m11s   192.168.0.249   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
</code></pre>

<p>But the moment I try to join a worker node to the master with <code>kubeadm join "$api_server_endpoint" --token "$token" --discovery-token-ca-cert-hash "$hash"</code>, all of the <code>kube-system</code> containers start to crash:</p>

<pre><code>sysadm@master$ sudo kubectl get pods --all-namespaces -o wide
NAMESPACE     NAME                      READY   STATUS             RESTARTS   AGE   IP              NODE                    NOMINATED NODE   READINESS GATES   
kube-system   calico-node-ntzn2         2/2     Running            0          10m   192.168.0.182   localhost.localdomain   &lt;none&gt;           &lt;none&gt;            
kube-system   coredns-fb8b8dccf-hqmn2   0/1     CrashLoopBackOff   2          10m   10.1.0.2        localhost.localdomain   &lt;none&gt;           &lt;none&gt;            
kube-system   coredns-fb8b8dccf-nfgr5   0/1     CrashLoopBackOff   1          10m   10.1.0.3        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-proxy-xgnlb          1/1     Running            0          10m   192.168.0.166   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
sysadm@master$ sudo kubectl get pods --all-namespaces -o wide
NAMESPACE     NAME                                            READY   STATUS             RESTARTS   AGE   IP              NODE                    NOMINATED NODE   READINESS GATES
kube-system   calico-node-ntzn2                               2/2     Running            0          11m   192.168.0.166   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-hqmn2                         0/1     CrashLoopBackOff   2          11m   10.1.0.2        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-nfgr5                         0/1     CrashLoopBackOff   2          11m   10.1.0.3        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   etcd-localhost.localdomain                      0/1     Pending            0          1s    &lt;none&gt;          localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-apiserver-localhost.localdomain            0/1     Pending            0          1s    &lt;none&gt;          localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-controller-manager-localhost.localdomain   0/1     Pending            0          1s    &lt;none&gt;          localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-proxy-xgnlb                                1/1     Running            0          11m   192.168.0.249   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
sysadm@master$ sudo kubectl get pods --all-namespaces -o wide
NAMESPACE     NAME                                   READY   STATUS    RESTARTS   AGE   IP              NODE                    NOMINATED NODE   READINESS GATES
kube-system   calico-node-ntzn2                      2/2     Running   0          11m   192.168.0.182   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-hqmn2                0/1     Running   3          11m   10.1.0.2        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-nfgr5                0/1     Running   2          11m   10.1.0.3        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-proxy-xgnlb                       1/1     Running   0          11m   192.168.0.166   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-scheduler-localhost.localdomain   0/1     Pending   0          0s    &lt;none&gt;          localhost.localdomain   &lt;none&gt;           &lt;none&gt;
sysadm@master$ sudo kubectl get pods --all-namespaces -o wide
NAMESPACE     NAME                      READY   STATUS    RESTARTS   AGE   IP              NODE                    NOMINATED NODE   READINESS GATES
kube-system   calico-node-ntzn2         2/2     Running   0          11m   192.168.0.182   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-hqmn2   1/1     Running   0          11m   10.1.0.2        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-nfgr5   1/1     Running   0          11m   10.1.0.3        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-proxy-xgnlb          1/1     Running   0          11m   192.168.0.166   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
sysadm@master$ sudo kubectl get pods --all-namespaces -o wide
NAMESPACE     NAME                                   READY   STATUS    RESTARTS   AGE   IP              NODE                    NOMINATED NODE   READINESS GATES
kube-system   calico-node-ntzn2                      2/2     Running   0          11m   192.168.0.166   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-hqmn2                0/1     Error     2          11m   10.1.0.2        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-nfgr5                1/1     Running   0          11m   10.1.0.3        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-proxy-xgnlb                       1/1     Running   0          11m   192.168.0.249   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-scheduler-localhost.localdomain   0/1     Pending   0          0s    &lt;none&gt;          localhost.localdomain   &lt;none&gt;           &lt;none&gt;
sysadm@master$ sudo kubectl get pods --all-namespaces -o wide
NAMESPACE     NAME                         READY   STATUS             RESTARTS   AGE   IP              NODE                    NOMINATED NODE   READINESS GATES
kube-system   calico-node-ntzn2            2/2     Running            0          11m   192.168.0.249   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-hqmn2      0/1     CrashLoopBackOff   2          11m   10.1.0.2        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-nfgr5      1/1     Running            0          11m   10.1.0.3        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   etcd-localhost.localdomain   0/1     Pending            0          1s    &lt;none&gt;          localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-proxy-xgnlb             1/1     Running            0          11m   192.168.0.166   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
sysadm@master$ sudo kubectl get pods --all-namespaces -o wide
NAMESPACE     NAME                                   READY   STATUS    RESTARTS   AGE   IP              NODE                    NOMINATED NODE   READINESS GATES
kube-system   calico-node-ntzn2                      2/2     Running   0          11m   192.168.0.182   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-hqmn2                0/1     Error     3          11m   10.1.0.2        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-nfgr5                0/1     Error     2          11m   10.1.0.3        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-apiserver-localhost.localdomain   0/1     Pending   0          0s    &lt;none&gt;          localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-proxy-xgnlb                       1/1     Running   0          11m   192.168.0.249   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
sysadm@master$ sudo kubectl get pods --all-namespaces -o wide
NAMESPACE     NAME                      READY   STATUS             RESTARTS   AGE   IP              NODE                    NOMINATED NODE   READINESS GATES
kube-system   calico-node-ntzn2         2/2     Running            0          11m   192.168.0.249   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-hqmn2   1/1     Running            0          11m   10.1.0.2        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-nfgr5   0/1     CrashLoopBackOff   2          11m   10.1.0.3        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-proxy-xgnlb          1/1     Running            0          11m   192.168.0.166   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
sysadm@master$ sudo kubectl get pods --all-namespaces -o wide
NAMESPACE     NAME                      READY   STATUS             RESTARTS   AGE   IP              NODE                    NOMINATED NODE   READINESS GATES
kube-system   calico-node-ntzn2         2/2     Running            0          11m   192.168.0.249   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-hqmn2   0/1     Running            3          11m   10.1.0.2        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   coredns-fb8b8dccf-nfgr5   0/1     CrashLoopBackOff   2          11m   10.1.0.3        localhost.localdomain   &lt;none&gt;           &lt;none&gt;
kube-system   kube-proxy-xgnlb          1/1     Running            0          11m   192.168.0.166   localhost.localdomain   &lt;none&gt;           &lt;none&gt;
</code></pre>

<p>Any ideas what might be going on?  How can I troubleshoot this?  I try to use <code>kubectl describe pods</code> but the pods keep on crashing and when I am able to get some information back, I don't see anything to lead me where to investigate next.</p>

<p>Sorry for the vague details.  If you can point me to where else to look, I can post more details or know where to investigate next.</p>

<p>Thank you for your time :)</p>

Zhao Li
<p>issue is with the hostname. check NODENAME column. it shows the host name as localhost.localdomain</p>

<p>update the hostname as k8s-master or master. it should work. each node also should have a unique hostname like node1, node2, node3 and so on</p>
