<p>In Kubernetes, I have a statefulset with a number of replicas.
I've set the updateStrategy to RollingUpdate.
I've set podManagementPolicy to Parallel.
My statefulset instances do not have a persistent volume claim -- I use the statefulset as a way to allocate ordinals 0..(N-1) to pods in a deterministic manner.</p>
<p>The main reason for this, is to keep availability for new requests while rolling out software updates (freshly built containers) while still allowing each container, and other services in the cluster, to &quot;know&quot; its ordinal.</p>
<p>The behavior I want, when doing a rolling update, is for the previous statefulset pods to linger while there are still long-running requests processing on them, but I want new traffic to go to the new pods in the statefulset (mapped by the ordinal) without a temporary outage.</p>
<p>Unfortunately, I don't see a way of doing this -- what am I missing?</p>
<p>Because I don't use volume claims, you might think I could use deployments instead, but I really do need each of the pods to have a deterministic ordinal, that:</p>
<ul>
<li>is unique at the point of dispatching new service requests (incoming HTTP requests, including public ingresses)</li>
<li>is discoverable by the pod itself</li>
<li>is persistent for the duration of the pod lifetime</li>
<li>is contiguous from 0 .. (N-1)</li>
</ul>
<p>The second-best option I can think of is using something like zookeeper or etcd to separately manage this property, using some of the traditional long-poll or leader-election mechanisms, but given that kubernetes already knows (or can know) about all the necessary bits, AND kubernetes service mapping knows how to steer incoming requests from old instances to new instances, that seems more redundant and complicated than necessary, so I'd like to avoid that.</p>

Jon Watte
<p>I assume that you need this for a <em>stateful workload</em>, a workload that e.g. requires writes. Otherwise you can use Deployments with multiple pods online for your shards. A key feature with StatefulSet is that they provide <strong>unique stable network identities</strong> for the instances.</p>
<blockquote>
<p>The behavior I want, when doing a rolling update, is for the previous statefulset pods to linger while there are still long-running requests processing on them, but I want new traffic to go to the new pods in the statefulset.</p>
</blockquote>
<p>This behavior is supported by Kubernetes pods. But you also need to implement support for it in your application.</p>
<ul>
<li>New traffic will not be sent to your &quot;old&quot; pods.</li>
<li>A <code>SIGTERM</code> signal will be sent to the pod - your application may want to listen to this and do some action.</li>
<li>After a <em>configurable</em> &quot;termination grace period&quot;, your pod will get killed.</li>
</ul>
<p>See <a href="https://cloud.google.com/blog/products/containers-kubernetes/kubernetes-best-practices-terminating-with-grace" rel="nofollow noreferrer">Kubernetes best practices: terminating with grace</a> for more info about pod termination.</p>
<p>Be aware that you should connect to <em>services</em> instead of directly to <em>pods</em> for this to work. E.g. you need to create <a href="https://kubernetes.io/docs/concepts/services-networking/service/#headless-services" rel="nofollow noreferrer">headless services</a> for the replicas in a <a href="https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/" rel="nofollow noreferrer">StatefulSet</a>.</p>
<p>If your clients are connecting to a specific <em>headless service</em>, e.g. <code>N</code>, this means that it will not be available for some times during upgrades. You need to decide if your clients should <strong>retry</strong> their connections during this time period <em>or</em> if they should connect to another <em>headless service</em> if <code>N</code> is not available.</p>
<p>If you are in a case where you need:</p>
<ul>
<li>stateful workload (e.g. support for write operations)</li>
<li>want high availability for your instances</li>
</ul>
<p>then you need a form of distributed system that does some form of replication/synchronization, e.g. using <a href="https://raft.github.io/" rel="nofollow noreferrer">raft</a> or a product that implements this. Such system is easiest deployed as a StatefulSet.</p>
