title: Kubernetes dial tcp myIP:10250: connect: no route to host
body: <p>I got Kubernetes Cluster with 1 master and 3 workers nodes.</p>
  <p>calico v3.7.3 kubernetes v1.16.0 installed via kubespray <a href="https://github.com/kubernetes-sigs/kubespray" rel="nofollow noreferrer">https://github.com/kubernetes-sigs/kubespray</a></p>
  <p>Before that, I normally deployed all the pods without any problems.</p>
  <p>I can't start a few pod (Ceph):</p>
  <pre><code>kubectl get all --namespace=ceph
  NAME                                 READY   STATUS             RESTARTS   AGE
  pod/ceph-cephfs-test                 0/1     Pending            0          162m
  pod/ceph-mds-665d849f4f-fzzwb        0/1     Pending            0          162m
  pod/ceph-mon-744f6dc9d6-jtbgk        0/1     CrashLoopBackOff   24         162m
  pod/ceph-mon-744f6dc9d6-mqwgb        0/1     CrashLoopBackOff   24         162m
  pod/ceph-mon-744f6dc9d6-zthpv        0/1     CrashLoopBackOff   24         162m
  pod/ceph-mon-check-6f474c97f-gjr9f   1/1     Running            0          162m
  
  
  NAME               TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
  service/ceph-mon   ClusterIP   None         &lt;none&gt;        6789/TCP   162m
  
  NAME                      DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR       AGE
  daemonset.apps/ceph-osd   0         0         0       0            0           node-type=storage   162m
  
  NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
  deployment.apps/ceph-mds         0/1     1            0           162m
  deployment.apps/ceph-mon         0/3     3            0           162m
  deployment.apps/ceph-mon-check   1/1     1            1           162m
  
  NAME                                       DESIRED   CURRENT   READY   AGE
  replicaset.apps/ceph-mds-665d849f4f        1         1         0       162m
  replicaset.apps/ceph-mon-744f6dc9d6        3         3         0       162m
  replicaset.apps/ceph-mon-check-6f474c97f   1         1         1       162m
  
  </code></pre>
  <p>But another obe is ok:</p>
  <pre><code>kubectl get pods -n kube-system
  NAME                                       READY   STATUS    RESTARTS   AGE
  calico-kube-controllers-6d57b44787-xlj89   1/1     Running   19         24d
  calico-node-dwm47                          1/1     Running   310        19d
  calico-node-hhgzk                          1/1     Running   15         24d
  calico-node-tk4mp                          1/1     Running   309        19d
  calico-node-w7zvs                          1/1     Running   312        19d
  coredns-74c9d4d795-jrxjn                   1/1     Running   0          2d23h
  coredns-74c9d4d795-psf2v                   1/1     Running   2          18d
  dns-autoscaler-7d95989447-7kqsn            1/1     Running   10         24d
  kube-apiserver-master                      1/1     Running   4          24d
  kube-controller-manager-master             1/1     Running   3          24d
  kube-proxy-9bt8m                           1/1     Running   2          19d
  kube-proxy-cbrcl                           1/1     Running   4          19d
  kube-proxy-stj5g                           1/1     Running   0          19d
  kube-proxy-zql86                           1/1     Running   0          19d
  kube-scheduler-master                      1/1     Running   3          24d
  kubernetes-dashboard-7c547b4c64-6skc7      1/1     Running   591        24d
  nginx-proxy-worker1                        1/1     Running   2          19d
  nginx-proxy-worker2                        1/1     Running   0          19d
  nginx-proxy-worker3                        1/1     Running   0          19d
  nodelocaldns-6t92x                         1/1     Running   2          19d
  nodelocaldns-kgm4t                         1/1     Running   0          19d
  nodelocaldns-xl8zg                         1/1     Running   0          19d
  nodelocaldns-xwlwk                         1/1     Running   12         24d
  tiller-deploy-8557598fbc-7f2w6             1/1     Running   0          131m
  
  </code></pre>
  <p>I use Centos 7:</p>
  <pre><code>NAME=&quot;CentOS Linux&quot;
  VERSION=&quot;7 (Core)&quot;
  ID=&quot;centos&quot;
  ID_LIKE=&quot;rhel fedora&quot;
  VERSION_ID=&quot;7&quot;
  PRETTY_NAME=&quot;CentOS Linux 7 (Core)&quot;
  ANSI_COLOR=&quot;0;31&quot;
  CPE_NAME=&quot;cpe:/o:centos:centos:7&quot;
  HOME_URL=&quot;https://www.centos.org/&quot;
  BUG_REPORT_URL=&quot;https://bugs.centos.org/&quot;
  
  CENTOS_MANTISBT_PROJECT=&quot;CentOS-7&quot;
  CENTOS_MANTISBT_PROJECT_VERSION=&quot;7&quot;
  REDHAT_SUPPORT_PRODUCT=&quot;centos&quot;
  REDHAT_SUPPORT_PRODUCT_VERSION=&quot;7&quot;
  </code></pre>
  <p>The error log:</p>
  <pre><code>Get https://10.2.67.203:10250/containerLogs/ceph/ceph-mon-744f6dc9d6-mqwgb/ceph-mon?tailLines=5000&amp;timestamps=true: dial tcp 10.2.67.203:10250: connect: no route to host
  </code></pre>
  <p>Maybe someone came across this and can help me? I will provide any additional information</p>
  <p>logs from pending pods:</p>
  <p>Warning  FailedScheduling  98s (x125 over 3h1m)  default-scheduler  0/4 nodes are available: 4 node(s) didn't match node selector.</p>