title: Kubernetes pod stuck in state=Terminating after node goes to status = NotReady?
body: <p>I have 3 node in k8s cluster, where all are masters, i.e. I have removed the <code>node-role.kubernetes.io/master</code> taint.</p>
  
  <p>I physically removed the network cable on <code>foo2</code>, so I have </p>
  
  <pre><code>kubectl get nodes
  NAME   STATUS     ROLES    AGE     VERSION
  foo1   Ready      master   3d22h   v1.13.5
  foo2   NotReady   master   3d22h   v1.13.5
  foo3   Ready      master   3d22h   v1.13.5
  </code></pre>
  
  <p>After several hours some of the pods are still in <code>STATUS = Terminating</code> though I think they should be in <code>Terminated</code> ?</p>
  
  <p>I read at <a href="https://www.bluematador.com/docs/troubleshooting/kubernetes-pod" rel="nofollow noreferrer">https://www.bluematador.com/docs/troubleshooting/kubernetes-pod</a></p>
  
  <blockquote>
    <p>In rare cases, it is possible for a pod to get stuck in the terminating state. This is detected by finding any pods where every container has been terminated, but the pod is still running. Usually, this is caused when a node in the cluster gets taken out of service abruptly, and the cluster scheduler and controller-manager do not clean up all of the pods on that node.     </p>
    
    <p>Solving this issue is as simple as manually deleting the pod using kubectl delete pod .</p>
  </blockquote>
  
  <p>The pod describe says if unreachable for 5 minutes will be tolerated ...</p>
  
  <pre><code>Conditions:
    Type              Status
    Initialized       True 
    Ready             False 
    ContainersReady   True 
    PodScheduled      True 
  Volumes:
    etcd-data:
      Type:        EmptyDir (a temporary directory that shares a pod's lifetime)
      Medium:      
      SizeLimit:   &lt;unset&gt;
  QoS Class:       BestEffort
  Node-Selectors:  &lt;none&gt;
  Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                   node.kubernetes.io/unreachable:NoExecute for 300s
  Events:          &lt;none&gt;
  </code></pre>
  
  <p>I have tried <code>kubectl delete pod etcd-lns4g5xkcw</code> which just hung, though forcing it <strong>does</strong> work as per this <a href="https://stackoverflow.com/questions/55935173/kubernetes-pods-stuck-with-in-terminating-state">answer</a> ...</p>
  
  <pre><code>kubectl delete pod etcd-lns4g5xkcw  --force=true --grace-period=0
  warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
  pod "etcd-lns4g5xkcw" force deleted
  </code></pre>
  
  <p>(1) Why is this happening ? Shouln't it change to terminated? </p>
  
  <p>(2) Where even is <code>STATUS = Terminating</code> coming from? At <a href="https://v1-13.docs.kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/" rel="nofollow noreferrer">https://v1-13.docs.kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/</a> I See only Waiting/Running/Terminated as the options ?</p>