<p>What is the best way to convert a kubernetes non-HA control plane into an HA control plane?</p>
<p>I have started with my cluster as a non-HA control plane - one master node and several worker nodes. The cluster is already running with a lots of services.</p>
<p>Now I would like to add additional master nodes to convert my cluster into a HA control plane. I have setup and configured a load balancer.
But I did not figure out how I can change the -control-plane-endpoint to my load balancer IP address for my existing master node.</p>
<p>Calling kubeadm results in the following error:</p>
<pre><code>sudo kubeadm init --control-plane-endpoint &quot;my-load-balancer:6443&quot; --upload-certs
[init] Using Kubernetes version: v1.20.1
[preflight] Running pre-flight checks
    [WARNING SystemVerification]: missing optional cgroups: hugetlb
error execution phase preflight: [preflight] Some fatal errors occurred:
    [ERROR Port-6443]: Port 6443 is in use
    [ERROR Port-10259]: Port 10259 is in use
    [ERROR Port-10257]: Port 10257 is in use
    [ERROR FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml]: /etc/kubernetes/manifests/kube-apiserver.yaml already exists
    [ERROR FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml]: /etc/kubernetes/manifests/kube-controller-manager.yaml already exists
    [ERROR FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml]: /etc/kubernetes/manifests/kube-scheduler.yaml already exists
    [ERROR FileAvailable--etc-kubernetes-manifests-etcd.yaml]: /etc/kubernetes/manifests/etcd.yaml already exists
    [ERROR Port-10250]: Port 10250 is in use
    [ERROR Port-2379]: Port 2379 is in use
    [ERROR Port-2380]: Port 2380 is in use
    [ERROR DirAvailable--var-lib-etcd]: /var/lib/etcd is not empty
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
</code></pre>
<p>The error message seems to be clear as my master is already running.
Is there a way how I can easily tell my existing master node to use the new load balancer to run as a HA control plane?</p>

<h2>Best solution in my opinion</h2>
<p>The best approach to <code>convert</code> a <strong>non-HA</strong> control plane to an <strong>HA</strong> control plane is to create a completely new <strong>HA</strong> control plane and after that to migrate all your applications there.</p>
<h2>Possible solution</h2>
<p>Below I will try to help you to achieve your goal but <strong>I do not recommend using this procedure on any cluster that will ever be considered as production cluster.</strong> It work for my scenario, it also might help you.</p>
<h3>Update the kube-apiserver certificate</h3>
<p>First of all, <strong>kube-apiserver</strong> uses a <code>certificate</code> to encrypt control plane traffic and this <code>certificate</code> have something known as <strong>SAN</strong> (Subject Alternative Name).
SAN is a list of IP addresses that you will use to access the API, so you need to add there IP address of your <code>LoadBalancer</code> and probably the <code>hostname</code> of your <code>LB</code> as well.</p>
<p>To do that, you have to get <code>kubeadm</code> configuration e.g. using command:</p>
<pre><code>   $ kubeadm config view &gt; kubeadm-config.yaml
</code></pre>
<p>and then add <code>certSANs</code> to <code>kubeadm-config.yaml</code> config file under <code>apiServer</code> section, it should looks like below example: (you may also need to add <code>controlPlaneEndpoint</code> to point to your <code>LB</code>).</p>
<pre><code>apiServer:
  certSANs:
  - &quot;192.168.0.2&quot; # your LB address
  - &quot;loadbalancer&quot; # your LB hostname
  extraArgs:
    authorization-mode: Node,RBAC
    ...
 controlPlaneEndpoint: &quot;loadbalancer&quot; # your LB DNS name or DNS CNAME
    ...
</code></pre>
<p>Now you can update <code>kube-apiserver</code> cert using:<br />
<strong>BUT please remember</strong> you must first delete/move your old <code>kube-apiserver</code> cert and key from <code>/etc/kubernetes/pki/</code> !</p>
<p>$ kubeadm init phase certs apiserver --config kubeadm-config.yaml.</p>
<p>Finally restart your <code>kube-apiserver</code>.</p>
<h3>Update the kubelet, the scheduler and the controller manager kubeconfig files</h3>
<p>Next step is to update the <code>kubelet</code>, <code>scheduler</code> and <code>controller manager</code> to communicate with the <code>kube-apiserver</code> using <code>LoadBalancer</code>.<br />
All three of these components use standard kubeconfig files:<br />
<code>/etc/kubernetes/kubelet.conf</code>, <code>/etc/kubernetes/scheduler.conf</code>, <code>/etc/kubernetes/controller-manager.conf</code> to communicate with <code>kube-apiserver</code>.<br />
The only thing to do is to edit the <code>server:</code> line to point to <code>LB</code> instead of <code>kube-apiserver</code> directly and then restart these components.</p>
<p>The <code>kubelet</code> is <code>systemd</code> service so to restart it use:</p>
<pre><code>systemctl restart kubelet 
</code></pre>
<p>the <code>controller manager</code> and <code>schedulers</code> are deployed as pods.</p>
<h3>Update the kube-proxy kubeconfig files</h3>
<p>Next it is time to update <code>kubeconfig</code> file for <code>kube-proxy</code> and same as before - the only thing to do is to edit the <code>server:</code> line to point to <code>LoadBalancer</code> instead of <code>kube-apiserver</code> directly.</p>
<p>This <code>kubeconfig</code> is in fact a <code>configmap</code>, so you can edit it directly using:</p>
<pre><code>$ kubectl edit cm kube-proxy -n kube-system
</code></pre>
<p>or first save it as manifest file:</p>
<pre><code>$ kubectl get cm kube-proxy -n kube-system -o yaml &gt; kube-proxy.yml 
</code></pre>
<p>and then apply changes.</p>
<p><strong>Don't forget</strong> to restart <code>kube-proxy</code> after these changes.</p>
<h3>Update the kubeadm-config configmap</h3>
<p>At the end upload new <code>kubeadm-config</code> <code>configmap</code> (with <code>certSANs</code> and <code>controlPlaneEndpoint</code> entries) to the cluster, it's especially important when you want to add new node to the cluster.</p>
<pre><code>$ kubeadm config upload from-file --config kubeadm-config.yaml
</code></pre>
<p>if command above doesn't work, try this:</p>
<pre><code>$ kubeadm upgrade apply --config kubeadm-config.yaml
</code></pre>
