<p>Using Kubernetes, we could control how many pods (app instances) are created with a new deployment rollout simultaneously. It's achievable by using properties <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#max-surge" rel="nofollow noreferrer">max surge</a> and <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#max-unavailable" rel="nofollow noreferrer">max unavailable</a>. If we have tens of app instances, the default configuration will rollout multiple new instances and at the same time, multiple instances will be terminating.
The most preferable configuration for us is changing that to <code>max surge = 1</code> and <code>max unavailable = 0</code> (reason - to achieve smooth Kafka rebalancing). In that case, only a single app instance will be started at a specific time, and up to a few instances will be terminating.</p>
<p>As I see, the deployment strategy if we have <code>max surge = N</code> and <code>max unavailable = 0</code> is the following:</p>
<ul>
<li>step 1: N new pods (the first batch) starting</li>
<li>step 2: N pods terminating, and N new  pods (the second batch) starting</li>
<li>step 3: N more pods terminating (and together with step 2, we could experience
up to 2N terminating pods),  and N new  pods (the third batch)
starting</li>
<li>step 4: same logic, and here we could experience more than 2N
pods still terminating</li>
<li>and so on...</li>
</ul>
<p>For our micro-service with total number of pods 30, with <code>max surge = 1</code> and <code>max unavailable = 0</code>, we have up to 3 terminating pods simultaneously, and with <code>max surge = 3</code> and <code>max unavailable = 0</code> - up to 7 terminating pods.</p>
<p><strong>Is it possible to control the maximum number of simultaneously terminating pods during deployment rollout?</strong> Let's say I want to see at most one pod in a terminating state. So until pod will not be shut down completely, no new pods started.</p>
<p><code>kubectl version</code>:</p>
<pre><code>Client Version: version.Info{Major:&quot;1&quot;, Minor:&quot;18&quot;, GitVersion:&quot;v1.18.10&quot;, GitCommit:&quot;62876fc6d93e891aa7fbe19771e6a6c03773b0f7&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2020-10-15T01:52:24Z&quot;, GoVersion:&quot;go1.13.15&quot;, Compiler:&quot;gc&quot;, Platform:&quot;linux/amd64&quot;}
Server Version: version.Info{Major:&quot;1&quot;, Minor:&quot;18&quot;, GitVersion:&quot;v1.18.10&quot;, GitCommit:&quot;62876fc6d93e891aa7fbe19771e6a6c03773b0f7&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2020-10-15T01:43:56Z&quot;, GoVersion:&quot;go1.13.15&quot;, Compiler:&quot;gc&quot;, Platform:&quot;linux/amd64&quot;}
</code></pre>

<p><code>maxUnavailable</code> and <code>maxSurge</code> is definitely the way to go. The steps you listed are on point, and it is how it's supposed to work.</p>
<p>What you are experiencing is most probably a bug. There is one similiar issue on GitHub (<a href="https://github.com/kubernetes/kubernetes/issues/99513" rel="nofollow noreferrer">#99513</a>) about Kubernetes not respecting <code>maxUnavailable</code> and <code>maxSurge</code>.</p>
<p>If you can, create new issue on GitHub, as yours is a bit different then aforementioned one.</p>
<hr />
<p><strong>EDIT</strong></p>
<p>I found another similiar issue (<a href="https://github.com/kubernetes/kubernetes/issues/95498" rel="nofollow noreferrer">#95498</a>) with a response, stating it's working as intended. <sup>[<a href="https://github.com/kubernetes/kubernetes/issues/95498#issuecomment-712508628" rel="nofollow noreferrer">source</a>]</sup></p>
<blockquote>
<p>When the Deployment controller performs a rolling update on a Deployment, it scales up/down ReplicaSets based on the <code>.spec.replicas </code>of the ReplicaSets it controls. When a ReplicaSet is scaled down, the ReplicaSet's <code>.spec.replicas</code> will be reduced and some of its Pods will be terminating, and the Deployment controller simply ignores those terminating Pods when calculating maxSurge / maxUnavailable.</p>
<p>Similarly, ReplicaSet controller ignores terminating Pods while doing operations, such as calculating its <code>.status.availableReplicas</code></p>
</blockquote>
<p>but I'm having my doubts. Creating new issue on GitHub, and getting a developer to look into it, is still the good option.</p>
