<p>I have a Kubernetes cluster of 3 nodes in Amazon EKS. It's running 3 pods of Cockroachdb in a StatefulSet. Now I want to use another instance type for all nodes of my cluster.
So my plan was this:</p>
<ol>
<li>Add 1 new node to the cluster, increase replicas in my StatefulSet to 4 and wait for the new Cockroachdb pod to fully sync.</li>
<li>Decommission and stop one of the old Cockroachdb nodes.</li>
<li>Decrease replicas of the StatefulSet back to 3 to get rid of one of the old pods.</li>
<li>Repeat steps 1-3 two more times.</li>
</ol>
<p>Obviously, that doesn't work because StatefulSet deletes most recent pods first when scaling down, so my new pod gets deleted instead of the old one.
I guess I could just create a new StatefulSet and make it use existing PVs, but that doesn't seem like the best solution for me. Is there any other way to do the migration?</p>

<p>You can consider make a copy of your ASG current launch template -&gt; upgrade the instance type of the copied template -&gt; point your ASG to use this new launch template -&gt; perform ASG instance refresh. Cluster of 3 nodes with minimum 90% of healthy % ensure only 1 instance will be replace at a time. Affected pod on the drained node will enter pending state for 5~10 mins and redeploy on the new node. This way you do not need to scale up StatefulSet un-necessary.</p>
