<p>There is a k8s single master node, I need to back it up and restore on a different server with different ip addresses. I googled this topic and found a solution -
<a href="https://elastisys.com/2018/12/10/backup-kubernetes-how-and-why/" rel="nofollow noreferrer">https://elastisys.com/2018/12/10/backup-kubernetes-how-and-why/</a></p>

<p>Everything looked easy; so, I followed the instruction and got a copy of the certificates and a snapshot of the etcd database. Then I used the second script to restore the node on a different server. It did not go well this time. It gave me a bunch of errors related to mismatching the certificates and server's local ip addresses.</p>

<p>As far as I understood, when a kubernetes cluster is initializing, it creates a set of certificates assigned to the original server's ip addresses and I cannot just back it up and restore somewhere else. </p>

<p>So, how to backup a k8s master node and restore it?</p>

Samuel
<p>Make sure that you added an extra flag to the kubeadm init command (<code>--ignore-preflight-errors=DirAvailable--var-lib-etcd</code>) to acknowledge that we want to use the pre-existing data.</p>

<p>Do the following steps:</p>

<ul>
<li>replace the IP address in all config files in <code>/etc/kubernetes</code></li>
<li>back up <code>/etc/kubernetes/pki</code></li>
<li>identify certs in <code>/etc/kubernetes/pki</code> that have the old IP address
as an alt name - <strong>1st</strong> step</li>
<li>delete both the cert and key for each of them (for me it was just
apiserver and etcd/peer)</li>
<li>regenerate the certs using kubeadm alpha phase certs - <strong>2nd</strong> step</li>
<li>identify configmap in the kube-system namespace that referenced
the old IP - <strong>3rd</strong> step</li>
<li>manually edit those configmaps</li>
<li>restart kubelet and docker (to force all containers to be
recreated)</li>
</ul>

<p><strong>1.</strong></p>

<pre><code>/etc/kubernetes/pki# for f in $(find -name "*.crt"); do openssl x509 -in $f -text -noout &gt; $f.txt; done
/etc/kubernetes/pki# grep -Rl 12\\.34\\.56\\.78 .
./apiserver.crt.txt
./etcd/peer.crt.txt
/etc/kubernetes/pki# for f in $(find -name "*.crt"); do rm $f.txt; done
</code></pre>

<p><strong>2.</strong></p>

<pre><code>/etc/kubernetes/pki# rm apiserver.crt apiserver.key
/etc/kubernetes/pki# kubeadm alpha phase certs apiserver
...
/etc/kubernetes/pki# rm etcd/peer.crt etcd/peer.key
/etc/kubernetes/pki# kubeadm alpha phase certs etcd-peer
</code></pre>

<p>...
<strong>3.</strong></p>

<pre><code>$ kubectl -n kube-system get cm -o yaml | less
...
$ kubectl -n kube-system edit cm ...
</code></pre>

<p>Take a look here: <a href="https://github.com/kubernetes/kubeadm/issues/338" rel="nofollow noreferrer">master-backup</a>.</p>

<p><strong>UPDATE:</strong></p>

<p>During replacing master nodes and changing IP you cannot contact the api-server to change the configmaps in step 4. Moreover if you have single master k8s cluster connection between worker nodes will be interrupted till new master will be up. </p>

<p>To ensure connection between master and worker nodes during master replacement you have to create <a href="https://searchdisasterrecovery.techtarget.com/definition/high-availability-cluster-HA-cluster" rel="nofollow noreferrer">HA cluster</a>.</p>

<p>The certificate is signed for {your-old-IP-here} and secure communication can't then happen to {your-new-ip-here}</p>

<p>You can add more IPs in the certificate in beforehand though...</p>

<p>The api-server certificate is signed for hostname kubernetes, so you can add that as an alias to the new IP address in <code>/etc/hosts</code> then do k<code>ubectl --server=https://kubernetes:6443 ....</code> .</p>
