title: Kubeadm Failed to create SubnetManager: error retrieving pod spec for kube-system
body: <p>No matter what I do it seems I cannot get rid of this problem.  I have installed Kubernetes using kubeadm many times quite successfully however adding a v1.16.0 node is giving me a heck of a headache. </p>
  
  <p>O/S: Ubuntu 18.04.3 LTS
  Kubernetes version: v1.16.0
  Kubeadm version: Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:34:01Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"</p>
  
  <p>A query of the cluster shows:</p>
  
  <pre><code>    NAME                  STATUS                     ROLES    AGE     VERSION
  kube-apiserver-1      Ready                      master   110d    v1.15.0
  kube-apiserver-2      Ready                      master   110d    v1.15.0
  kube-apiserver-3      Ready                      master   110d    v1.15.0
  kube-node-1           Ready                      &lt;none&gt;   110d    v1.15.0
  kube-node-2           Ready                      &lt;none&gt;   110d    v1.15.0
  kube-node-3           Ready                      &lt;none&gt;   110d    v1.15.0
  kube-node-4           Ready                      &lt;none&gt;   110d    v1.16.0
  kube-node-5           Ready,SchedulingDisabled   &lt;none&gt;   3m28s   v1.16.0
  kube-node-databases   Ready                      &lt;none&gt;   110d    v1.15.0
  </code></pre>
  
  <p>I have temporarily disabled scheduling to the node until I can fix this problem.  A query of the pod status in the kube-system namespace shows the problem:</p>
  
  <p>$ kubectl -n kube-system get pods</p>
  
  <pre><code>NAME                                       READY   STATUS             RESTARTS   AGE
  coredns-fb8b8dccf-55zjs                    1/1     Running            128        21d
  coredns-fb8b8dccf-kzrpc                    1/1     Running            144        21d
  kube-flannel-ds-amd64-29xp2                1/1     Running            11         110d
  kube-flannel-ds-amd64-hp7nq                1/1     Running            14         110d
  kube-flannel-ds-amd64-hvdpf                0/1     CrashLoopBackOff   5          8m28s
  kube-flannel-ds-amd64-jhhlk                1/1     Running            11         110d
  kube-flannel-ds-amd64-k6dzc                1/1     Running            2          110d
  kube-flannel-ds-amd64-lccxl                1/1     Running            21         110d
  kube-flannel-ds-amd64-nnn7g                1/1     Running            14         110d
  kube-flannel-ds-amd64-shss5                1/1     Running            7          110d
  </code></pre>
  
  <p>kubectl -n kube-system logs -f kube-flannel-ds-amd64-hvdpf </p>
  
  <pre><code>I1002 01:13:22.136379       1 main.go:514] Determining IP address of default interface
  I1002 01:13:22.136823       1 main.go:527] Using interface with name ens3 and address 192.168.5.46
  I1002 01:13:22.136849       1 main.go:544] Defaulting external address to interface address (192.168.5.46)
  E1002 01:13:52.231471       1 main.go:241] Failed to create SubnetManager: error retrieving pod spec for 'kube-system/kube-flannel-ds-amd64-hvdpf': Get https://10.96.0.1:443/api/v1/namespaces/kube-system/pods/kube-flannel-ds-amd64-hvdpf: dial tcp 10.96.0.1:443: i/o timeout
  </code></pre>
  
  <p>Although I had a few hits on iptables issues and kernel routing I don't understand why previous versions have installed without a hitch but this version is giving me such a problem.</p>
  
  <p>I have installed this node and destroyed it quite a few times yet the result is always the same.</p>
  
  <p>Anyone else having this issue or has a solution?</p>