<p>I have an EKS cluster running a StatefulSet using EC2 nodes and EBS volumes for the storageclass.  I want to move a pod of the StatefulSet from node 1 to node 2.  I drain node 1 like so:</p>
<pre><code>kubectl drain --ignore-daemonsets --delete-emptydir-data node1
</code></pre>
<p>The problem is the pod doesn't come up on node 2, because the PV has been created in us-east-1a and can't be attached to node 2 which is in us-east-1b (cross-zone issue described here: <a href="https://stackoverflow.com/a/55514852/1259990">https://stackoverflow.com/a/55514852/1259990</a>).</p>
<p>When I describe the pod, I get the following scheduling error:</p>
<blockquote>
<p>1 node(s) had volume node affinity conflict</p>
</blockquote>
<p>I'm wondering if I can recreate the PV in us-east-1b without having to delete/redeploy the StatefulSet.  If I were to delete the PV from my cluster (and possibly the PVC as well):</p>
<pre><code>kubectl delete pv pv-in-us-east-1a
</code></pre>
<p>Would the StatefulSet recreate the PV in the correct zone, if node2 is the only schedulable node?  If not, is there another way to accomplish this without deleting/recreating the full StatefulSet?  The data on the PV is not important and doesn't need to be saved.</p>
<p>(I would just try to delete the PV, but I don't actually want to bring down this particular service if the PV doesn't get recreated.)</p>

blindsnowmobile
<p>What you need to do is</p>
<ol>
<li>is labels on your nodes per zone (you do that using the autoscaling group tags)</li>
<li>on statefulset that requires PV you use a node selector for a specific zone: i.e. us-east-1</li>
</ol>
<p>That statefulset will be locked to a specific AZ but you will avoid this problem in the future.</p>
