title: Kubelet service is not running. It seems like the kubelet isn't running or healthy
body: <p>I have configured 1 master 2 workers.
  after installation successfully kubernetes. It is OK with worker1 joining cluster but I can not join worker2 to the cluster
  because kubelet service is not running. <strong>It seems like the kubelet isn't running or healthy</strong></p>
  <p><strong>sudo kubectl get nodes:</strong></p>
  <blockquote>
  <p>NAME      STATUS     ROLES                  AGE     VERSION<br />
  master1   Ready      control-plane,master   23m     v1.22.2<br />
  node1     NotReady                    4m13s   v1.22.2</p>
  </blockquote>
  <p>I want to know why the kubelet service is not running.</p>
  <p><strong>Here kubelet logs.</strong></p>
  <pre><code>The start-up result is RESULT.
  Dec 04 20:21:26 node2 kubelet[25435]: Flag --network-plugin has been deprecated, will be removed along with dockershim.
  Dec 04 20:21:26 node2 kubelet[25435]: Flag --network-plugin has been deprecated, will be removed along with dockershim.
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.659131   25435 server.go:440] &quot;Kubelet version&quot; kubeletVersion=&quot;v1.22.2&quot;
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.659587   25435 server.go:868] &quot;Client rotation is on, will bootstrap in background&quot;
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.678863   25435 certificate_store.go:130] Loading cert/key pair from &quot;/var/lib/kubelet/pki/kubelet-client-current.pem&quot;.
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.684321   25435 dynamic_cafile_content.go:155] &quot;Starting controller&quot; name=&quot;client-ca-bundle::/etc/kubernetes/pki/ca.crt&quot;
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.728096   25435 server.go:687] &quot;--cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /&quot;
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.728320   25435 container_manager_linux.go:280] &quot;Container manager verified user specified cgroup-root exists&quot; cgroupRoot=[]
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.728388   25435 container_manager_linux.go:285] &quot;Creating Container Manager object based on Node Config&quot; nodeConfig={RuntimeCgroupsName:
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.729329   25435 topology_manager.go:133] &quot;Creating topology manager with policy per scope&quot; topologyPolicyName=&quot;none&quot; topologyScopeName=&quot;c
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.729345   25435 container_manager_linux.go:320] &quot;Creating device plugin manager&quot; devicePluginEnabled=true
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.729367   25435 state_mem.go:36] &quot;Initialized new in-memory state store&quot;
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.729408   25435 kubelet.go:314] &quot;Using dockershim is deprecated, please consider using a full-fledged CRI implementation&quot;
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.729430   25435 client.go:78] &quot;Connecting to docker on the dockerEndpoint&quot; endpoint=&quot;unix:///var/run/docker.sock&quot;
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.729441   25435 client.go:97] &quot;Start docker client with request timeout&quot; timeout=&quot;2m0s&quot;
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.744324   25435 docker_service.go:566] &quot;Hairpin mode is set but kubenet is not enabled, falling back to HairpinVeth&quot; hairpinMode=promiscu
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.744354   25435 docker_service.go:242] &quot;Hairpin mode is set&quot; hairpinMode=hairpin-veth
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.744554   25435 cni.go:239] &quot;Unable to update cni config&quot; err=&quot;no networks found in /etc/cni/net.d&quot;
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.750011   25435 cni.go:239] &quot;Unable to update cni config&quot; err=&quot;no networks found in /etc/cni/net.d&quot;
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.750260   25435 docker_service.go:257] &quot;Docker cri networking managed by the network plugin&quot; networkPluginName=&quot;cni&quot;
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.753050   25435 cni.go:239] &quot;Unable to update cni config&quot; err=&quot;no networks found in /etc/cni/net.d&quot;
  Dec 04 20:21:26 node2 kubelet[25435]: I1204 20:21:26.764080   25435 docker_service.go:264] &quot;Docker Info&quot; dockerInfo=&amp;{ID:4UUR:AFJU:SXYE:5IRP:6G6B:SFDY:H3AA:D5ZB:JSDO:GXVQ:UYNG:POJY Containe
  Dec 04 20:21:26 node2 kubelet[25435]: E1204 20:21:26.765777   25435 server.go:294] &quot;Failed to run kubelet&quot; err=&quot;failed to run Kubelet: misconfiguration: kubelet cgroup driver: \&quot;systemd\&quot; i
  Dec 04 20:21:26 node2 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
  Dec 04 20:21:26 node2 systemd[1]: kubelet.service: Failed with result 'exit-code'.
  
  </code></pre>
  <p><strong>kubeadm join logs:</strong></p>
  <pre><code>I1204 20:27:56.222794   29796 join.go:405] [preflight] found NodeName empty; using OS hostname as NodeName
  I1204 20:27:56.223032   29796 initconfiguration.go:116] detected and using CRI socket: /var/run/dockershim.sock
  [preflight] Running pre-flight checks
  I1204 20:27:56.223834   29796 preflight.go:92] [preflight] Running general checks
  I1204 20:27:56.225983   29796 checks.go:245] validating the existence and emptiness of directory /etc/kubernetes/manifests
  I1204 20:27:56.226133   29796 checks.go:282] validating the existence of file /etc/kubernetes/kubelet.conf
  I1204 20:27:56.226271   29796 checks.go:282] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf
  I1204 20:27:56.226408   29796 checks.go:106] validating the container runtime
  I1204 20:27:56.282374   29796 checks.go:132] validating if the &quot;docker&quot; service is enabled and active
  I1204 20:27:56.300100   29796 checks.go:331] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables
  I1204 20:27:56.300279   29796 checks.go:331] validating the contents of file /proc/sys/net/ipv4/ip_forward
  I1204 20:27:56.300580   29796 checks.go:649] validating whether swap is enabled or not
  I1204 20:27:56.300738   29796 checks.go:372] validating the presence of executable conntrack
  I1204 20:27:56.301009   29796 checks.go:372] validating the presence of executable ip
  I1204 20:27:56.301613   29796 checks.go:372] validating the presence of executable iptables
  I1204 20:27:56.301801   29796 checks.go:372] validating the presence of executable mount
  I1204 20:27:56.302057   29796 checks.go:372] validating the presence of executable nsenter
  I1204 20:27:56.302384   29796 checks.go:372] validating the presence of executable ebtables
  I1204 20:27:56.302473   29796 checks.go:372] validating the presence of executable ethtool
  I1204 20:27:56.302569   29796 checks.go:372] validating the presence of executable socat
  I1204 20:27:56.302610   29796 checks.go:372] validating the presence of executable tc
  I1204 20:27:56.303072   29796 checks.go:372] validating the presence of executable touch
  I1204 20:27:56.303472   29796 checks.go:520] running all checks
  I1204 20:27:56.372402   29796 checks.go:403] checking whether the given node name is valid and reachable using net.LookupHost
  I1204 20:27:56.373211   29796 checks.go:618] validating kubelet version
  I1204 20:27:56.467792   29796 checks.go:132] validating if the &quot;kubelet&quot; service is enabled and active
  I1204 20:27:56.485715   29796 checks.go:205] validating availability of port 10250
  I1204 20:27:56.486624   29796 checks.go:282] validating the existence of file /etc/kubernetes/pki/ca.crt
  I1204 20:27:56.487016   29796 checks.go:432] validating if the connectivity type is via proxy or direct
  I1204 20:27:56.487841   29796 join.go:475] [preflight] Discovering cluster-info
  I1204 20:27:56.488260   29796 token.go:80] [discovery] Created cluster-info discovery client, requesting info from &quot;192.168.1.53:6443&quot;
  I1204 20:27:56.520182   29796 token.go:118] [discovery] Requesting info from &quot;192.168.1.53:6443&quot; again to validate TLS against the pinned public key
  I1204 20:27:56.530589   29796 token.go:135] [discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server &quot;192.168.1.53:6443&quot;
  I1204 20:27:56.530702   29796 discovery.go:52] [discovery] Using provided TLSBootstrapToken as authentication credentials for the join process
  I1204 20:27:56.530924   29796 join.go:489] [preflight] Fetching init configuration
  I1204 20:27:56.531171   29796 join.go:534] [preflight] Retrieving KubeConfig objects
  [preflight] Reading configuration from the cluster...
  [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
  I1204 20:27:56.549808   29796 interface.go:431] Looking for default routes with IPv4 addresses
  I1204 20:27:56.549913   29796 interface.go:436] Default route transits interface &quot;enp0s3&quot;
  I1204 20:27:56.550259   29796 interface.go:208] Interface enp0s3 is up
  I1204 20:27:56.550564   29796 interface.go:256] Interface &quot;enp0s3&quot; has 2 addresses :[192.168.1.50/24 fe80::a00:27ff:fe7e:db8b/64].
  I1204 20:27:56.550644   29796 interface.go:223] Checking addr  192.168.1.50/24.
  I1204 20:27:56.550887   29796 interface.go:230] IP found 192.168.1.50
  I1204 20:27:56.550955   29796 interface.go:262] Found valid IPv4 address 192.168.1.50 for interface &quot;enp0s3&quot;.
  I1204 20:27:56.551237   29796 interface.go:442] Found active IP 192.168.1.50
  I1204 20:27:56.563573   29796 preflight.go:103] [preflight] Running configuration dependant checks
  I1204 20:27:56.563872   29796 controlplaneprepare.go:219] [download-certs] Skipping certs download
  I1204 20:27:56.565399   29796 kubelet.go:112] [kubelet-start] writing bootstrap kubelet config file at /etc/kubernetes/bootstrap-kubelet.conf
  I1204 20:27:56.569613   29796 kubelet.go:120] [kubelet-start] writing CA certificate at /etc/kubernetes/pki/ca.crt
  I1204 20:27:56.572216   29796 kubelet.go:141] [kubelet-start] Checking for an existing Node in the cluster with name &quot;node2&quot; and status &quot;Ready&quot;
  I1204 20:27:56.576685   29796 kubelet.go:155] [kubelet-start] Stopping the kubelet
  [kubelet-start] Writing kubelet configuration to file &quot;/var/lib/kubelet/config.yaml&quot;
  [kubelet-start] Writing kubelet environment file with flags to file &quot;/var/lib/kubelet/kubeadm-flags.env&quot;
  [kubelet-start] Starting the kubelet
  [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
  I1204 20:28:01.956734   29796 kubelet.go:190] [kubelet-start] preserving the crisocket information for the node
  I1204 20:28:01.956911   29796 patchnode.go:31] [patchnode] Uploading the CRI Socket information &quot;/var/run/dockershim.sock&quot; to the Node API object &quot;node2&quot; as an annotation
  I1204 20:28:01.957066   29796 cert_rotation.go:137] Starting client certificate rotation controller
  [kubelet-check] Initial timeout of 40s passed.
  [kubelet-check] It seems like the kubelet isn't running or healthy.
  [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get &quot;http://localhost:10248/healthz&quot;: dial tcp 127.0.0.1:10248: connect: connection refused.
  [kubelet-check] It seems like the kubelet isn't running or healthy.
  [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get &quot;http://localhost:10248/healthz&quot;: dial tcp 127.0.0.1:10248: connect: connection refused.
  [kubelet-check] It seems like the kubelet isn't running or healthy.
  [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get &quot;http://localhost:10248/healthz&quot;: dial tcp 127.0.0.1:10248: connect: connection refused.
  [kubelet-check] It seems like the kubelet isn't running or healthy.
  [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get &quot;http://localhost:10248/healthz&quot;: dial tcp 127.0.0.1:10248: connect: connection refused.
  
  </code></pre>