<p>Has anyone faced a scenario where the pod gets evicted from one node(ie Node A) and then is immeditaley scheduled on another node(ie Node B).
But the kubelet of Node A keeps complaining about not being able to delete the container of the pod. The container is up &amp; is not killed.</p>

<pre><code>Aug 30 20:29:36 staging-node-4 kubelet[2173]: I0830 20:29:36.358238    2173 kubelet_pods.go:1073] Killing unwanted pod "thanos-compactor-0"
Aug 30 20:29:36 staging-node-4 kubelet[2173]: I0830 20:29:36.362581    2173 kuberuntime_container.go:559] Killing container "docker://b22287cd406c3fe9eff4ff2df1792c6f84b5b92d001359f05ea73f8788715609" with 30 second grace period
Aug 30 20:29:36 staging-node-4 kubelet[2173]: E0830 20:29:36.363416    2173 kuberuntime_container.go:71] Can't make a ref to pod "thanos-compactor-0_thanos(23096418-c7b2-11e9-9be7-005056bc2883)", container thanos: selfLink was empty, can't make reference
Aug 30 20:29:38 staging-node-4 kubelet[2173]: I0830 20:29:38.362191    2173 kubelet_pods.go:1073] Killing unwanted pod "thanos-compactor-0"
Aug 30 20:29:38 staging-node-4 kubelet[2173]: I0830 20:29:38.366884    2173 kuberuntime_container.go:559] Killing container "docker://b22287cd406c3fe9eff4ff2df1792c6f84b5b92d001359f05ea73f8788715609" with 30 second grace period
Aug 30 20:29:38 staging-node-4 kubelet[2173]: E0830 20:29:38.367901    2173 kuberuntime_container.go:71] Can't make a ref to pod "thanos-compactor-0_thanos(23096418-c7b2-11e9-9be7-005056bc2883)", container thanos: selfLink was empty, can't make reference
Aug 30 20:29:38 staging-node-4 kubelet[2173]: I0830 20:29:38.997510    2173 setters.go:72] Using node IP: "172.17.21.146"
Aug 30 20:29:40 staging-node-4 kubelet[2173]: I0830 20:29:40.364388    2173 kubelet_pods.go:1073] Killing unwanted pod "thanos-compactor-0"
Aug 30 20:29:40 staging-node-4 kubelet[2173]: I0830 20:29:40.368226    2173 kuberuntime_container.go:559] Killing container "docker://b22287cd406c3fe9eff4ff2df1792c6f84b5b92d001359f05ea73f8788715609" with 30 second grace period
Aug 30 20:29:40 staging-node-4 kubelet[2173]: E0830 20:29:40.369235    2173 kuberuntime_container.go:71] Can't make a ref to pod "thanos-compactor-0_thanos(23096418-c7b2-11e9-9be7-005056bc2883)", container thanos: selfLink was empty, can't make reference
</code></pre>

<p>b22287cd is the docker containerid that is not deleted.</p>

<p>I experienced this issue a few weeks ago. 
Pods were evicted from a node and sometimes were 'stuck' in terminating state. </p>

<p>Check your pod state by running <code>kubectl -n &lt;NAMESPACE&gt; get pod</code></p>

<p>Using a <code>kubectl delete pod &lt;PODNAME&gt; --grace-period=0 --force --namespace &lt;NAMESPACE&gt;</code> resolved the problem.</p>
