<p>I had set-up the Kubernetes cluster using kubeadm and it was working fine. I was able to deploy pods and service. I had tried restarting the cluster and it used to come up without any issues. Suddenly, after a few days, I had to restart my machine(hosting clusters on vms) and the cluster started not coming up. Not able to understand why is this started happening now. Can somebody help me please in troubleshooting this issue?</p>

<p>This is how my cluster looks:</p>

<pre><code> Node                         Hostname         IP 
 master                       master-kube      192.168.50.10
 worker-node-1                node-1-kube      192.168.50.11
 worker-node-2                node-2-kube      192.168.50.12  
</code></pre>

<p>All nodes(ubuntu bionic) hosted on a virtual box.</p>

<p>More details on cluster:</p>

<pre><code>kubelet = 1.16.0-00
kubeadm = 1.16.0-00
kubectl = 1.16.0-00
docker = 18.06.2-ce, build 6d37f41
pod-network = calico-latest
command used to init cluster = kubeadm init --apiserver-advertise-address="192.168.50.10" --apiserver-cert-extra-sans="192.168.50.10" --pod-network-cidr=172.16.0.0/16

</code></pre>

<p>I would like to highlight the errors I am getting currently to restart the kubelet:</p>

<pre><code>
kubelet[19946]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubel
kubelet[19946]: Flag --resolv-conf has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet
kubelet[19946]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubel
kubelet[19946]: Flag --resolv-conf has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet
kubelet[19946]: I1221 05:37:49.631489   19946 server.go:410] Version: v1.16.0
kubelet[19946]: I1221 05:37:49.631636   19946 plugins.go:100] No cloud provider specified.
kubelet[19946]: I1221 05:37:49.631649   19946 server.go:773] Client rotation is on, will bootstrap in background
kubelet[19946]: I1221 05:37:49.633624   19946 certificate_store.go:129] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem".
kubelet[19946]: I1221 05:37:49.658641   19946 server.go:644] --cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /
kubelet[19946]: I1221 05:37:49.659118   19946 container_manager_linux.go:265] container manager verified user specified cgroup-root exists: []
kubelet[19946]: I1221 05:37:49.659269   19946 container_manager_linux.go:270] Creating Container Manager object based on Node Config: {RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime
kubelet[19946]: I1221 05:37:49.659508   19946 fake_topology_manager.go:29] [fake topologymanager] NewFakeManager
kubelet[19946]: I1221 05:37:49.659604   19946 container_manager_linux.go:305] Creating device plugin manager: true
kubelet[19946]: I1221 05:37:49.659735   19946 fake_topology_manager.go:39] [fake topologymanager] AddHintProvider HintProvider:  &amp;{kubelet.sock /var/lib/kubelet/device-plugins/ map[] {0 0} &lt;nil&gt; {{} [0 0 0]} 0x
kubelet[19946]: I1221 05:37:49.659850   19946 state_mem.go:36] [cpumanager] initializing new in-memory state store
kubelet[19946]: I1221 05:37:49.660030   19946 state_mem.go:84] [cpumanager] updated default cpuset: ""
kubelet[19946]: I1221 05:37:49.660127   19946 state_mem.go:92] [cpumanager] updated cpuset assignments: "map[]"
kubelet[19946]: I1221 05:37:49.660220   19946 fake_topology_manager.go:39] [fake topologymanager] AddHintProvider HintProvider:  &amp;{{0 0} 0x799d338 10000000000 0xc0009ee9c0 &lt;nil&gt; &lt;nil&gt; &lt;nil&gt; &lt;nil&gt; map[memory:{{1
kubelet[19946]: I1221 05:37:49.660363   19946 kubelet.go:287] Adding pod path: /etc/kubernetes/manifests
kubelet[19946]: I1221 05:37:49.660502   19946 kubelet.go:312] Watching apiserver
kubelet[19946]: E1221 05:37:49.663227   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&amp;resourceVersion
kubelet[19946]: E1221 05:37:49.663829   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
kubelet[19946]: I1221 05:37:49.665047   19946 client.go:75] Connecting to docker on unix:///var/run/docker.sock
kubelet[19946]: I1221 05:37:49.665066   19946 client.go:104] Start docker client with request timeout=2m0s
kubelet[19946]: E1221 05:37:49.665772   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
kubelet[19946]: W1221 05:37:49.667035   19946 docker_service.go:563] Hairpin mode set to "promiscuous-bridge" but kubenet is not enabled, falling back to "hairpin-veth"
kubelet[19946]: I1221 05:37:49.667144   19946 docker_service.go:240] Hairpin mode set to "hairpin-veth"
kubelet[19946]: I1221 05:37:49.694036   19946 docker_service.go:255] Docker cri networking managed by cni
kubelet[19946]: I1221 05:37:49.701825   19946 docker_service.go:260] Docker Info: &amp;{ID:OF2B:RLPC:I6RY:MIMU:H5S4:3GBP:QB6T:IXQF:TMTB:7JBZ:XY2O:ICQO Containers:28 ContainersRunning:6 ContainersPaused:0 Containers
kubelet[19946]: I1221 05:37:49.702214   19946 docker_service.go:273] Setting cgroupDriver to cgroupfs
kubelet[19946]: I1221 05:37:49.711797   19946 remote_runtime.go:59] parsed scheme: ""
 kubelet[19946]: I1221 05:37:49.711813   19946 remote_runtime.go:59] scheme "" not registered, fallback to default scheme
kubelet[19946]: I1221 05:37:49.711835   19946 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/var/run/dockershim.sock 0  &lt;nil&gt;}] &lt;nil&gt;}
kubelet[19946]: I1221 05:37:49.711841   19946 clientconn.go:577] ClientConn switching balancer to "pick_first"
kubelet[19946]: I1221 05:37:49.711856   19946 remote_image.go:50] parsed scheme: ""
kubelet[19946]: I1221 05:37:49.711860   19946 remote_image.go:50] scheme "" not registered, fallback to default scheme
Dec 21 05:37:49 master-kube kubelet[19946]: I1221 05:37:49.711866   19946 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/var/run/dockershim.sock 0  &lt;nil&gt;}] &lt;nil&gt;}
kubelet[19946]: I1221 05:37:49.711870   19946 clientconn.go:577] ClientConn switching balancer to "pick_first"
kubelet[19946]: E1221 05:37:49.916339   19946 aws_credentials.go:77] while getting AWS credentials NoCredentialProviders: no valid providers in chain. Deprecated.
kubelet[19946]:         For verbose messaging see aws.Config.CredentialsChainVerboseErrors
 kubelet[19946]: I1221 05:37:49.920329   19946 kuberuntime_manager.go:207] Container runtime docker initialized, version: 18.06.2-ce, apiVersion: 1.38.0
kubelet[19946]: I1221 05:37:49.925361   19946 server.go:1065] Started kubelet
Dec 21 05:37:49 master-kube kubelet[19946]: E1221 05:37:49.925583   19946 kubelet.go:1302] Image garbage collection failed once. Stats initialization may not have completed yet: failed to get imageFs info: unable to find data in memory ca
kubelet[19946]: E1221 05:37:49.927007   19946 event.go:246] Unable to write event: 'Post https://192.168.50.10:6443/api/v1/namespaces/default/events: dial tcp 192.168.50.10:6443: connect: connection refused' (m
kubelet[19946]: I1221 05:37:49.928287   19946 server.go:145] Starting to listen on 0.0.0.0:10250
kubelet[19946]: I1221 05:37:49.931484   19946 server.go:354] Adding debug handlers to kubelet server.
kubelet[19946]: I1221 05:37:49.935058   19946 fs_resource_analyzer.go:64] Starting FS ResourceAnalyzer
kubelet[19946]: I1221 05:37:49.935868   19946 status_manager.go:156] Starting to sync pod status with apiserver
kubelet[19946]: I1221 05:37:49.936434   19946 kubelet.go:1822] Starting kubelet main sync loop.
kubelet[19946]: I1221 05:37:49.936965   19946 kubelet.go:1839] skipping pod synchronization - [container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]
kubelet[19946]: I1221 05:37:49.943766   19946 volume_manager.go:249] Starting Kubelet Volume Manager
kubelet[19946]: E1221 05:37:49.955051   19946 controller.go:135] failed to ensure node lease exists, will retry in 200ms, error: Get https://192.168.50.10:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-l
kubelet[19946]: I1221 05:37:49.960391   19946 desired_state_of_world_populator.go:131] Desired state populator starts to run
kubelet[19946]: E1221 05:37:49.960555   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: Get https://192.168.50.10:6443/apis/storage.k8s.io/v1beta1/csidriver
kubelet[19946]: E1221 05:37:49.960939   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: Get https://192.168.50.10:6443/apis/node.k8s.io/v1beta1/runtimecl
kubelet[19946]: I1221 05:37:49.977365   19946 clientconn.go:104] parsed scheme: "unix"
kubelet[19946]: I1221 05:37:49.977587   19946 clientconn.go:104] scheme "unix" not registered, fallback to default scheme
kubelet[19946]: I1221 05:37:49.977801   19946 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0  &lt;nil&gt;}] &lt;nil&gt;}
kubelet[19946]: I1221 05:37:49.977941   19946 clientconn.go:577] ClientConn switching balancer to "pick_first"
kubelet[19946]: W1221 05:37:49.988778   19946 cni.go:328] CNI failed to retrieve network namespace path: **cannot find network namespace for the terminated container "2fe393eaa9c070dcc24651a27fa01e421ada7a8b20a4d**
kubelet[19946]: W1221 05:37:49.991163   19946 docker_sandbox.go:394] **failed to read pod IP from plugin/docker: networkPlugin cni failed on the status hook for pod "coredns-5644d7b6d9-j8f7f_kube-system": CNI fai**
kubelet[19946]: W1221 05:37:49.994413   19946 docker_sandbox.go:394] failed to read pod IP from plugin/docker: networkPlugin cni failed on the status hook for pod "coredns-5644d7b6d9-j8f7f_kube-system": CNI fai
kubelet[19946]: I1221 05:37:50.039352   19946 kubelet.go:1839] skipping pod synchronization - container runtime status check may not have completed yet
kubelet[19946]: E1221 05:37:50.045922   19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: I1221 05:37:50.046091   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: I1221 05:37:50.047491   19946 kubelet_node_status.go:72] Attempting to register node master-kube
kubelet[19946]: E1221 05:37:50.047805   19946 kubelet_node_status.go:94] Unable to register node "master-kube" with API server: Post https://192.168.50.10:6443/api/v1/nodes: dial tcp 192.168.50.10:6443: connect
kubelet[19946]: I1221 05:37:50.048908   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: I1221 05:37:50.050080   19946 cpu_manager.go:161] [cpumanager] starting with none policy
kubelet[19946]: I1221 05:37:50.050184   19946 cpu_manager.go:162] [cpumanager] reconciling every 10s
kubelet[19946]: I1221 05:37:50.050283   19946 policy_none.go:42] [cpumanager] none policy: Start
kubelet[19946]: I1221 05:37:50.052514   19946 plugin_manager.go:116] Starting Kubelet Plugin Manager
kubelet[19946]: E1221 05:37:50.054259   19946 eviction_manager.go:246] eviction manager: failed to get summary stats: failed to get node info: node "master-kube" not found
kubelet[19946]: 2019-12-21 05:37:50.059 [INFO][20107] k8s.go 228: Using Calico IPAM
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.155714   19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: E1221 05:37:50.156669   19946 controller.go:135] failed to ensure node lease exists, will retry in 400ms, error: Get https://192.168.50.10:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-l
kubelet[19946]: I1221 05:37:50.254696   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: I1221 05:37:50.254929   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: E1221 05:37:50.257253   19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: I1221 05:37:50.264235   19946 kubelet_node_status.go:72] Attempting to register node master-kube
kubelet[19946]: E1221 05:37:50.267581   19946 kubelet_node_status.go:94] Unable to register node "master-kube" with API server: Post https://192.168.50.10:6443/api/v1/nodes: dial tcp 192.168.50.10:6443: connect
kubelet[19946]: I1221 05:37:50.278488   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: I1221 05:37:50.289042   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: I1221 05:37:50.298498   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: W1221 05:37:50.307965   19946 pod_container_deletor.go:75] Container "6d39199a688542a9336b33ed6e7d370ea6320ad47a2b17da800b0954bc985bec" not found in pod's containers
kubelet[19946]: W1221 05:37:50.308035   19946 pod_container_deletor.go:75] Container "d97a9bcd4aa63b11db422d92fed678a90878ab94528b3e6c1c7fd99200c972a7" not found in pod's containers
kubelet[19946]: E1221 05:37:50.361156   19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: I1221 05:37:50.362961   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-local-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/68aa1
kubelet[19946]: I1221 05:37:50.363626   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "k8s-certs" (UniqueName: "kubernetes.io/host-path/f15776993ab30bc91b546a2d4f3
kubelet[19946]: I1221 05:37:50.364328   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "ca-certs" (UniqueName: "kubernetes.io/host-path/68aa12790236de85ba80bddfd434
kubelet[19946]: I1221 05:37:50.364504   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "k8s-certs" (UniqueName: "kubernetes.io/host-path/68aa12790236de85ba80bddfd43
kubelet[19946]: I1221 05:37:50.364802   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/68aa1279023
kubelet[19946]: I1221 05:37:50.365575   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "etc-ca-certificates" (UniqueName: "kubernetes.io/host-path/f15776993ab30bc91
kubelet[19946]: I1221 05:37:50.366108   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "kubeconfig" (UniqueName: "kubernetes.io/host-path/4e1bd6e5b41d60d13135315758
kubelet[19946]: I1221 05:37:50.366614   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/f15776993ab
kubelet[19946]: I1221 05:37:50.367265   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "etcd-certs" (UniqueName: "kubernetes.io/host-path/cf6cdea881ad085c694fbe5bf3
kubelet[19946]: I1221 05:37:50.367499   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "etcd-data" (UniqueName: "kubernetes.io/host-path/cf6cdea881ad085c694fbe5bf37
kubelet[19946]: I1221 05:37:50.368086   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "ca-certs" (UniqueName: "kubernetes.io/host-path/f15776993ab30bc91b546a2d4f36
kubelet[19946]: I1221 05:37:50.368745   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "kubeconfig" (UniqueName: "kubernetes.io/host-path/f15776993ab30bc91b546a2d4f
kubelet[19946]: I1221 05:37:50.369349   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-local-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/f1577
kubelet[19946]: I1221 05:37:50.369922   19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "etc-ca-certificates" (UniqueName: "kubernetes.io/host-path/68aa12790236de85b
kubelet[19946]: E1221 05:37:50.461604   19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: E1221 05:37:50.598762   19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: E1221 05:37:50.598807   19946 controller.go:135] failed to ensure node lease exists, will retry in 800ms, error: Get https://192.168.50.10:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-l
kubelet[19946]: E1221 05:37:50.667616   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&amp;resourceVersion
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.667813   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
Dec 21 05:37:50 master-kube kubelet[19946]: I1221 05:37:50.669813   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.670999   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
Dec 21 05:37:50 master-kube kubelet[19946]: I1221 05:37:50.678853   19946 kubelet_node_status.go:72] Attempting to register node master-kube
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.680121   19946 kubelet_node_status.go:94] Unable to register node "master-kube" with API server: Post https://192.168.50.10:6443/api/v1/nodes: dial tcp 192.168.50.10:6443: connect
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.701314   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.802348   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.903471   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.994146   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: Get https://192.168.50.10:6443/apis/node.k8s.io/v1beta1/runtimecl
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.994146   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: Get https://192.168.50.10:6443/apis/storage.k8s.io/v1beta1/csidriver
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.004535   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.105943   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.207194   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.320505   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.401092   19946 controller.go:135] failed to ensure node lease exists, will retry in 1.6s, error: Get https://192.168.50.10:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-le
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.424687   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: I1221 05:37:51.485948   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
Dec 21 05:37:51 master-kube kubelet[19946]: I1221 05:37:51.493301   19946 kubelet_node_status.go:72] Attempting to register node master-kube
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.493752   19946 kubelet_node_status.go:94] Unable to register node "master-kube" with API server: Post https://192.168.50.10:6443/api/v1/nodes: dial tcp 192.168.50.10:6443: connect
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.524956   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.625893   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.669061   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&amp;resourceVersion
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.671281   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.672457   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.726204   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.827363   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.929008   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.996179   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: Get https://192.168.50.10:6443/apis/node.k8s.io/v1beta1/runtimecl
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.997820   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: Get https://192.168.50.10:6443/apis/storage.k8s.io/v1beta1/csidriver
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.035831   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.136937   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.238053   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.338740   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.439243   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.549008   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.649594   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.672587   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&amp;resourceVersion
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.673694   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.674519   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.749925   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.850243   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.950601   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.997582   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: Get https://192.168.50.10:6443/apis/node.k8s.io/v1beta1/runtimecl
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.999199   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: Get https://192.168.50.10:6443/apis/storage.k8s.io/v1beta1/csidriver
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.002688   19946 controller.go:135] failed to ensure node lease exists, will retry in 3.2s, error: Get https://192.168.50.10:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-le
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.051016   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: I1221 05:37:53.094012   19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
Dec 21 05:37:53 master-kube kubelet[19946]: I1221 05:37:53.097332   19946 kubelet_node_status.go:72] Attempting to register node master-kube
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.098210   19946 kubelet_node_status.go:94] Unable to register node "master-kube" with API server: Post https://192.168.50.10:6443/api/v1/nodes: dial tcp 192.168.50.10:6443: connect
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.152044   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.253059   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.353765   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.454334   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.554929   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.655913   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.675733   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&amp;resourceVersion
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.678185   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.678624   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.756190   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.856636   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.965559   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.004697   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: Get https://192.168.50.10:6443/apis/node.k8s.io/v1beta1/runtimecl
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.005008   19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: Get https://192.168.50.10:6443/apis/storage.k8s.io/v1beta1/csidriver
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.070989   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.171403   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.272286   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.372573   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.474665   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.574953   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.675738   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.677084   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&amp;resourceVersion
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.680293   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.681313   19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.782388   19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.883757   19946 kubelet.go:2267] node "master-kube" not found

</code></pre>

<p>docker ps command on master node:
<a href="https://i.stack.imgur.com/GEyJd.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/GEyJd.png" alt="enter image description here"></a></p>

Sudhir Kumar
<p>For better visibility of this post I am posting the solution that worked for this OP as <code>Community Wiki</code>.</p>
<p>The issue was resolved by completing tip posted by <code>@suren</code> who adviced to remove  <code>var/lib/kubelet</code> and re-install kubelet.</p>
<p>Some additional troubleshooting steps can be found in Kubernetes Documentation - <a href="https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/" rel="nofollow noreferrer">Troubleshooting kubeadm</a>.</p>
<p>In some cases also restarting of <code>kubelet</code> and <code>docker service</code> might help.</p>
<pre><code>sudo service docker restart
sudo systemctl restart kubelet
</code></pre>
