<p>I have a Kubernetes cluster of 3 masters and 2 nodes in VM cloude on CentOS7:</p>
<pre><code>[root@kbn-mst-02 ~]# kubectl get nodes
NAME         STATUS   ROLES    AGE    VERSION
kbn-mst-01   Ready    master   15d    v1.18.3
kbn-mst-02   Ready    master   14d    v1.18.3
kbn-mst-03   Ready    master   14d    v1.18.3
kbn-wn-01    Ready    &lt;none&gt;   25h    v1.18.5
kbn-wn-02    Ready    &lt;none&gt;   150m   v1.18.5
</code></pre>
<p>If I turn off kbn-mst-03 (212.46.30.7), then kbn-wn-01 and kbn-wn-02 get status <code>NotReady</code>:</p>
<pre><code>[root@kbn-mst-02 ~]# kubectl get nodes
NAME         STATUS     ROLES    AGE    VERSION
kbn-mst-01   Ready      master   15d    v1.18.3
kbn-mst-02   Ready      master   14d    v1.18.3
kbn-mst-03   NotReady   master   14d    v1.18.3
kbn-wn-01    NotReady   &lt;none&gt;   25h    v1.18.5
kbn-wn-02    NotReady   &lt;none&gt;   154m   v1.18.5
</code></pre>
<p>The log on kbn-wn-02 shows lost connection to 212.46.30.7:</p>
<pre><code>Jul  3 09:28:10 kbn-wn-02 kubelet: E0703 09:28:10.295233   12339 kubelet_node_status.go:402] Error updating node status, will retry: error getting node &quot;kbn-wn-02&quot;: Get https://212.46.30.7:6443/api/v1/nodes/kbn-wn-02?resourceVersion=0&amp;timeout=10s: context deadline exceede
</code></pre>
<p>Turning off other masters doesn't change the status of nodes.</p>
<p>Why does <code>kbn-wn-02</code> have a hard bind to <code>kbn-mst-03 (212.46.30.7)</code> and how can I change it?</p>

Valeriy Zavalishin
<p>Currently your worker nodes only know about <code>kbn-mst-03</code> master and when that master is turned off the kubelet on worker nodes can not send health status and metrics of the worker node to master node kbn-mst-03<code>and hence you see worker nodes as</code>NotReady`. This is also the reason why turning off other masters does not change the status of nodes because they are not known and contacted at all by kubelet of worker nodes.</p>
<p>You should use a load balancer in-front of the masters and use the load balancer endpoint while creating the worker nodes. Then if one master is turned off other master nodes will be able to handle requests because the load balancer will stop sending traffic to failed master and route traffic to other master.</p>
<p>How can you change the hard bind to one master and move to use load balancer endpoint will depend on what tool you used to setup the kubernetes cluster. If you are using kubeadm then you can specify a load balancer endpoint in <code>kubeadm init</code> in master nodes and use that endpoint in <code>kubeadm join</code> in worker nodes.</p>
<p>From kubeadm docs <a href="https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#considerations-about-apiserver-advertise-address-and-controlplaneendpoint" rel="nofollow noreferrer">here</a></p>
<blockquote>
<p><code>--control-plane-endpoint</code> can be used to set the shared endpoint for all control-plane nodes.</p>
<p><code>--control-plane-endpoint</code> allows both IP addresses and DNS names that can map to IP addresses. Please contact your network administrator to
evaluate possible solutions with respect to such mapping.</p>
<p>Here is an example mapping:</p>
<p><code>192.168.0.102 cluster-endpoint</code></p>
</blockquote>
<blockquote>
<p>Where <code>192.168.0.102</code> is the IP address of this node and cluster-endpoint is a custom DNS name that maps to
this IP. This will allow you to pass
<code>--control-plane-endpoint=cluster-endpoint</code> to <code>kubeadm init</code> and pass the same DNS name to <code>kubeadm join</code>. Later you can modify cluster-endpoint
to point to the address of your load-balancer in an high availability
scenario.</p>
</blockquote>
