<p>I added node to my cluster via <code>kubeadm join</code></p>

<p>All seems ok!</p>

<p>I see this node on my dashboard and healtz seems OK.</p>

<p>But if i start to deploy something i get error like whis one:</p>

<p><code>Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "1229e227176dfefd55c0728b10045bff158d99430843aca44c8957c2850127f2" network for pod "mobsf-76dd9dc786-zjjlx": NetworkPlugin cni failed to set up pod "" network: Get https://[10.233.0.1]:443/api/v1/namespaces/default: dial tcp 10.233.0.1:443: i/o timeout</code></p>

<p>I use calico as plugin</p>

<p>OS: Centos 7
Kernel:5.3.6-1.</p>

<p>NODES:
<code>kubectl get nodes
NAME                   STATUS   ROLES    AGE    VERSION
master                 Ready    master   143d   v1.15.3
s001dev-ceph-node-01   Ready    worker   40d    v1.15.3
s001dev-ceph-node-02   Ready    worker   40d    v1.15.3
s001dev-ceph-node-03   Ready    worker   40d    v1.15.3
worker1                Ready    worker   137d   v1.15.3
worker2                Ready    worker   137d   v1.15.3
worker3                Ready    worker   137d   v1.15.3
</code>
Calico:</p>

<p>``</p>

<pre><code>kubectl get all --all-namespaces | grep -i calico
kube-system   pod/calico-kube-controllers-6d57b44787-xlj89          1/1     Running             35         143d
kube-system   pod/calico-node-flhx4                                 1/1     Running             5          40d
kube-system   pod/calico-node-hhgzk                                 1/1     Running             30         143d
kube-system   pod/calico-node-qp2dr                                 1/1     Running             6          40d
kube-system   pod/calico-node-t698j                                 1/1     Running             183        40d
kube-system   pod/calico-node-tk4mp                                 1/1     Running             323        137d
kube-system   pod/calico-node-vg5qb                                 1/1     Running             7          40d
kube-system   pod/calico-node-w7zvs                                 1/1     Running             325        137d



kube-system   daemonset.apps/calico-node        7         7         7       7            7           &lt;none&gt;                        143d

kube-system   deployment.apps/calico-kube-controllers        1/1     1            1           143d

kube-system   replicaset.apps/calico-kube-controllers-6d57b44787       1         1         1       143d
</code></pre>

<p>Calico logs from first pod pod/calico-node-flhx4:</p>

<p><code>2020-02-03 06:55:38.094 [INFO][90] xdp_state.go 559: Finished XDP resync. family=4 resyncDuration=18.636208ms
2020-02-03 06:55:38.094 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=18.958571
2020-02-03 06:55:41.046 [INFO][90] int_dataplane.go 907: Applying dataplane updates
2020-02-03 06:55:41.046 [INFO][90] ipsets.go 223: Asked to resync with the dataplane on next update. family="inet"
2020-02-03 06:55:41.046 [INFO][90] ipsets.go 306: Resyncing ipsets with dataplane. family="inet"
2020-02-03 06:55:41.050 [INFO][90] ipsets.go 356: Finished resync family="inet" numInconsistenciesFound=0 resyncDuration=3.51651ms
2020-02-03 06:55:41.050 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=3.949963
2020-02-03 06:55:51.369 [INFO][90] int_dataplane.go 907: Applying dataplane updates
2020-02-03 06:55:51.369 [INFO][90] ipsets.go 223: Asked to resync with the dataplane on next update. family="inet"
2020-02-03 06:55:51.370 [INFO][90] ipsets.go 306: Resyncing ipsets with dataplane. family="inet"
2020-02-03 06:55:51.373 [INFO][90] ipsets.go 356: Finished resync family="inet" numInconsistenciesFound=0 resyncDuration=3.50255ms
2020-02-03 06:55:51.373 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=4.002705
2020-02-03 06:56:00.293 [INFO][90] int_dataplane.go 907: Applying dataplane updates
2020-02-03 06:56:00.293 [INFO][90] table.go 740: Invalidating dataplane cache ipVersion=0x4 reason="refresh timer" table="nat"
2020-02-03 06:56:00.295 [INFO][90] table.go 460: Loading current iptables state and checking it is correct. ipVersion=0x4 table="nat"
2020-02-03 06:56:00.305 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=11.403867
2020-02-03 06:56:01.433 [INFO][90] int_dataplane.go 907: Applying dataplane updates
2020-02-03 06:56:01.433 [INFO][90] ipsets.go 223: Asked to resync with the dataplane on next update. family="inet"
2020-02-03 06:56:01.433 [INFO][90] ipsets.go 306: Resyncing ipsets with dataplane. family="inet"
2020-02-03 06:56:01.437 [INFO][90] ipsets.go 356: Finished resync family="inet" numInconsistenciesFound=0 resyncDuration=3.505643ms
2020-02-03 06:56:01.437 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=3.87718
2020-02-03 06:56:10.788 [INFO][90] int_dataplane.go 907: Applying dataplane updates
2020-02-03 06:56:10.788 [INFO][90] table.go 740: Invalidating dataplane cache ipVersion=0x4 reason="refresh timer" table="raw"
2020-02-03 06:56:10.790 [INFO][90] table.go 460: Loading current iptables state and checking it is correct. ipVersion=0x4 table="raw"
2020-02-03 06:56:10.793 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=5.292443
2020-02-03 06:56:12.341 [INFO][90] int_dataplane.go 907: Applying dataplane updates
2020-02-03 06:56:12.342 [INFO][90] ipsets.go 223: Asked to resync with the dataplane on next update. family="inet"
2020-02-03 06:56:12.342 [INFO][90] ipsets.go 306: Resyncing ipsets with dataplane. family="inet"
2020-02-03 06:56:12.344 [INFO][90] ipsets.go 356: Finished resync family="inet" numInconsistenciesFound=0 resyncDuration=1.878346ms
2020-02-03 06:56:12.344 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=2.1785340000000004
2020-02-03 06:56:21.277 [INFO][90] int_dataplane.go 907: Applying dataplane updates
2020-02-03 06:56:21.277 [INFO][90] table.go 740: Invalidating dataplane cache ipVersion=0x4 reason="refresh timer" table="filter"
2020-02-03 06:56:21.281 [INFO][90] table.go 460: Loading current iptables state and checking it is correct. ipVersion=0x4 table="filter"
2020-02-03 06:56:21.286 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=8.782817
2020-02-03 06:56:22.914 [INFO][90] int_dataplane.go 907: Applying dataplane updates
2020-02-03 06:56:22.914 [INFO][90] ipsets.go 223: Asked to resync with the dataplane on next update. family="inet"
2020-02-03 06:56:22.914 [INFO][90] ipsets.go 306: Resyncing ipsets with dataplane. family="inet"
2020-02-03 06:56:22.917 [INFO][90] ipsets.go 356: Finished resync family="inet" numInconsistenciesFound=0 resyncDuration=2.961011ms
2020-02-03 06:56:22.917 [INFO][90] int_dataplane.go 921: Finished applying updates to dataplane. msecToApply=3.528314</code></p>

<p>From nodes worker1 and worker2 i can ping 10.233.0.1 (API), but cant at new node worker3:
<code>ping 10.233.0.1
PING 10.233.0.1 (10.233.0.1) 56(84) bytes of data.
From 10.254.0.4 icmp_seq=1 Time to live exceeded
From 10.254.0.4 icmp_seq=2 Time to live exceeded
</code></p>

<p>I also noticed the absence calico network interfaces on promlem node</p>

<p>I fixed this issue! Problems was that nginx-proxy pod on problems node was in pending statyus</p>
