title: How to properly recover a K8s cluster after reboot?
body: <p>I have a K8s cluster that was working properly but because of power failure, all the nodes got rebooted.</p>
  
  <p>At the moment I have some problem recovering the master (and other nodes):</p>
  
  <ol>
  <li><code>sudo systemctl kubelet status</code> is returning <code>Unknown operation kubelet.</code> but when I run <code>kubeadm init ...</code> (the command that I set up the cluster with) it returns:</li>
  </ol>
  
  <pre><code>error execution phase preflight: [preflight] Some fatal errors occurred:
      [ERROR Port-6443]: Port 6443 is in use
      [ERROR Port-10251]: Port 10251 is in use
      [ERROR Port-10252]: Port 10252 is in use
      [ERROR FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml]: /etc/kubernetes/manifests/kube-apiserver.yaml already exists
      [ERROR FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml]: /etc/kubernetes/manifests/kube-controller-manager.yaml already exists
      [ERROR FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml]: /etc/kubernetes/manifests/kube-scheduler.yaml already exists
      [ERROR FileAvailable--etc-kubernetes-manifests-etcd.yaml]: /etc/kubernetes/manifests/etcd.yaml already exists
      [ERROR Port-10250]: Port 10250 is in use
      [ERROR Port-2379]: Port 2379 is in use
      [ERROR Port-2380]: Port 2380 is in use
      [ERROR DirAvailable--var-lib-etcd]: /var/lib/etcd is not empty
  </code></pre>
  
  <p>and when I checked those ports I can see that kubelet and other K8s components are using them:</p>
  
  <pre><code>~/k8s-multi-node$ sudo lsof -i :10251
  COMMAND     PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
  kube-sche 26292 root    3u  IPv6 104933      0t0  TCP *:10251 (LISTEN)
  
  ~/k8s-multi-node$ sudo lsof -i :10252
  COMMAND     PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
  kube-cont 26256 root    3u  IPv6 115541      0t0  TCP *:10252 (LISTEN)
  
  ~/k8s-multi-node$ sudo lsof -i :10250
  COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
  kubelet 24781 root   27u  IPv6 106821      0t0  TCP *:10250 (LISTEN)
  </code></pre>
  
  <p>I tried to kill them but they start to use those ports again.</p>
  
  <ol start="2">
  <li>My second problem is because of the power failure my machines don't have access to internet at the moment.</li>
  </ol>
  
  <p>So what is the proper way to recover such a cluster? Do I need to remove kubelet and all the otehr components and install them again?</p>