<p>Trying to provision k8s cluster on 3 Debian 10 VMs with kubeadm.</p>
<p>All vms have 2 network interfaces, eth0 as public interface with static ip, eth1 as local interface with static ips in 192.168.0.0/16:</p>
<ul>
<li>Master: 192.168.1.1</li>
<li>Node1: 192.168.2.1</li>
<li>Node2: 192.168.2.2</li>
</ul>
<p>All nodes have interconnect between them.</p>
<p><code>ip a</code> from master host:</p>
<pre><code># ip a
1: lo: &lt;LOOPBACK,UP,LOWER_UP&gt; mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:52:70:53:d5:12 brd ff:ff:ff:ff:ff:ff
    inet XXX.XXX.244.240/24 brd XXX.XXX.244.255 scope global dynamic eth0
       valid_lft 257951sec preferred_lft 257951sec
    inet6 2a01:367:c1f2::112/48 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::252:70ff:fe53:d512/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:95:af:b0:8c:c4 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.1/16 brd 192.168.255.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::295:afff:feb0:8cc4/64 scope link 
       valid_lft forever preferred_lft forever
</code></pre>
<p>Master node is initialized fine with:</p>
<pre><code>kubeadm init --upload-certs --apiserver-advertise-address=192.168.1.1 --apiserver-cert-extra-sans=192.168.1.1,XXX.XXX.244.240 --pod-network-cidr=10.40.0.0/16 -v=5
</code></pre>
<p><a href="https://pastebin.com/raw/FfENpPnL" rel="nofollow noreferrer">Output</a></p>
<p>But when I join worker nodes kube-api is not reachable:</p>
<pre><code>kubeadm join 192.168.1.1:6443 --token 7bl0in.s6o5kyqg27utklcl --discovery-token-ca-cert-hash sha256:7829b6c7580c0c0f66aa378c9f7e12433eb2d3b67858dd3900f7174ec99cda0e -v=5
</code></pre>
<p><a href="https://pastebin.com/raw/ZE2rDNTY" rel="nofollow noreferrer">Output</a></p>
<p>Netstat from master:</p>
<pre><code># netstat -tupn | grep :6443
tcp        0      0 192.168.1.1:43332       192.168.1.1:6443        TIME_WAIT   -                   
tcp        0      0 192.168.1.1:41774       192.168.1.1:6443        ESTABLISHED 5362/kube-proxy     
tcp        0      0 192.168.1.1:41744       192.168.1.1:6443        ESTABLISHED 5236/kubelet        
tcp        0      0 192.168.1.1:43376       192.168.1.1:6443        TIME_WAIT   -                   
tcp        0      0 192.168.1.1:43398       192.168.1.1:6443        TIME_WAIT   -                   
tcp        0      0 192.168.1.1:41652       192.168.1.1:6443        ESTABLISHED 4914/kube-scheduler 
tcp        0      0 192.168.1.1:43448       192.168.1.1:6443        TIME_WAIT   -                   
tcp        0      0 192.168.1.1:43328       192.168.1.1:6443        TIME_WAIT   -                   
tcp        0      0 192.168.1.1:43452       192.168.1.1:6443        TIME_WAIT   -                   
tcp        0      0 192.168.1.1:43386       192.168.1.1:6443        TIME_WAIT   -                   
tcp        0      0 192.168.1.1:43350       192.168.1.1:6443        TIME_WAIT   -                   
tcp        0      0 192.168.1.1:41758       192.168.1.1:6443        ESTABLISHED 5182/kube-controlle 
tcp        0      0 192.168.1.1:43306       192.168.1.1:6443        TIME_WAIT   -                   
tcp        0      0 192.168.1.1:43354       192.168.1.1:6443        TIME_WAIT   -                   
tcp        0      0 192.168.1.1:43296       192.168.1.1:6443        TIME_WAIT   -                   
tcp        0      0 192.168.1.1:43408       192.168.1.1:6443        TIME_WAIT   -                   
tcp        0      0 192.168.1.1:41730       192.168.1.1:6443        ESTABLISHED 5182/kube-controlle 
tcp        0      0 192.168.1.1:41738       192.168.1.1:6443        ESTABLISHED 4914/kube-scheduler 
tcp        0      0 192.168.1.1:43444       192.168.1.1:6443        TIME_WAIT   -                   
tcp6       0      0 192.168.1.1:6443        192.168.1.1:41730       ESTABLISHED 5094/kube-apiserver 
tcp6       0      0 192.168.1.1:6443        192.168.1.1:41744       ESTABLISHED 5094/kube-apiserver 
tcp6       0      0 192.168.1.1:6443        192.168.1.1:41738       ESTABLISHED 5094/kube-apiserver 
tcp6       0      0 192.168.1.1:6443        192.168.1.1:41652       ESTABLISHED 5094/kube-apiserver 
tcp6       0      0 ::1:6443                ::1:42862               ESTABLISHED 5094/kube-apiserver 
tcp6       0      0 192.168.1.1:6443        192.168.1.1:41758       ESTABLISHED 5094/kube-apiserver 
tcp6       0      0 ::1:42862               ::1:6443                ESTABLISHED 5094/kube-apiserver 
tcp6       0      0 192.168.1.1:6443        192.168.1.1:41774       ESTABLISHED 5094/kube-apiserver 
</code></pre>
<p>Pods from master:</p>
<pre><code># kubectl --kubeconfig=/etc/kubernetes/admin.conf get pods -n kube-system -o wide
NAME                                              READY   STATUS    RESTARTS   AGE   IP                   NODE                      NOMINATED NODE   READINESS GATES
coredns-558bd4d5db-8qhhl                          0/1     Pending   0          12m   &lt;none&gt;               &lt;none&gt;                    &lt;none&gt;           &lt;none&gt;
coredns-558bd4d5db-9hj7z                          0/1     Pending   0          12m   &lt;none&gt;               &lt;none&gt;                    &lt;none&gt;           &lt;none&gt;
etcd-cloud604486.fastpipe.io                      1/1     Running   0          12m   2a01:367:c1f2::112   cloud604486.fastpipe.io   &lt;none&gt;           &lt;none&gt;
kube-apiserver-cloud604486.fastpipe.io            1/1     Running   0          12m   2a01:367:c1f2::112   cloud604486.fastpipe.io   &lt;none&gt;           &lt;none&gt;
kube-controller-manager-cloud604486.fastpipe.io   1/1     Running   0          12m   2a01:367:c1f2::112   cloud604486.fastpipe.io   &lt;none&gt;           &lt;none&gt;
kube-proxy-dzd42                                  1/1     Running   0          12m   2a01:367:c1f2::112   cloud604486.fastpipe.io   &lt;none&gt;           &lt;none&gt;
kube-scheduler-cloud604486.fastpipe.io            1/1     Running   0          12m   2a01:367:c1f2::112   cloud604486.fastpipe.io   &lt;none&gt;           &lt;none&gt;
</code></pre>
<p>All vms have this kernel parameters set:</p>
<ul>
<li><code>{ name: 'vm.swappiness', value: '0' }</code></li>
<li><code>{ name: 'net.bridge.bridge-nf-call-iptables', value: '1' }</code></li>
<li><code>{ name: 'net.bridge.bridge-nf-call-ip6tables', value: '1'}</code></li>
<li><code>{ name: 'net.ipv4.ip_forward', value: 1 }</code></li>
<li><code>{ name: 'net.ipv6.conf.all.forwarding', value: 1}</code></li>
</ul>
<p>br_netfilter kernel module active and iptables set to legacy mode (via alternatives)</p>
<p>Am I missing something?</p>

Alex Smith
<p>The reason for your issues is that the TLS connection between the components has to be secured. From the <code>kubelet</code> point of view this will be safe if the <code>Api-server</code> certificate will contain in alternative names the IP of the server that we want to connect to. You can notice yourself that you only add to <code>SANs</code> one IP address.</p>
<p>How can you fix this? There are two ways:</p>
<ol>
<li><p>Use the <code>--discovery-token-unsafe-skip-ca-verification</code> flag with your kubeadm join command from your node.</p>
</li>
<li><p>Add the IP address from the second <code>NIC</code> to <code>SANs</code> api certificate at the cluster initialization phase (kubeadm init)</p>
</li>
</ol>
<p>For more reading you check this directly related PR <a href="https://github.com/kubernetes/kubernetes/pull/93264" rel="nofollow noreferrer">#93264</a> which was introduced in kubernetes 1.19.</p>
