<p>I am trying to achieve the exactly-once delivery on Kafka using Spring-Kafka on Kubernetes.
As far as I understood, the transactional-ID must be set on the producer and it should be the same across restarts, as stated here <a href="https://stackoverflow.com/a/52304789/3467733">https://stackoverflow.com/a/52304789/3467733</a>.</p>
<p>The problem arises using this semantic on Kubernetes. <strong>How can you get a consistent ID?</strong></p>
<p>To solve this problem I implementend a Spring boot application, let's call it &quot;Replicas counter&quot; that checks, through the Kubernetes API, how many pods there are with the same name as the caller, so I have a counter for every pod replica.</p>
<p>For example, suppose I want to deploy a Pod, let's call it <code>APP-1</code>.</p>
<p>This app does the following:</p>
<ol>
<li>It perfoms a <code>GET</code> to the Replicas-Counter passing the pod-name as parameter.</li>
<li>The replicas-counter calls the Kubernetes API in order to check how many pods there are with that pod name. So it does a a +1 and returns it to the caller. I also need to count not-ready pods (think about a first deploy, they couldn't get the ID if I wasn't checking for not-ready pods).</li>
<li>The APP-1 gets the id and will use it as the transactional-id</li>
</ol>
<p>But, as you can see a problem could arise when performing rolling updates, for example:</p>
<p>Suppose we have 3 pods:</p>
<p>At the beginning we have:</p>
<ul>
<li>app-1: transactional-id-1</li>
<li>app-2: transactional-id-2</li>
<li>app-3: transactional-id-3</li>
</ul>
<p>So, during a rolling update we would have:</p>
<ul>
<li><p>old-app-1: transactional-id-1</p>
</li>
<li><p>old-app-2: transactional-id-2</p>
</li>
<li><p>old-app-3: transactional-id-3</p>
</li>
<li><p>new-app-3: transactional-id-4 (Not ready, waiting to be ready)</p>
<p>New-app-3 goes ready, so Kubernetes brings down the Old-app-3. So time to continue the rolling update.</p>
</li>
<li><p>old-app-1: transactional-id-1</p>
</li>
<li><p>old-app-2: transactional-id-2</p>
</li>
<li><p>new-app-3: transactional-id-4</p>
</li>
<li><p>new-app-2: transactional-id-4 (Not ready, waiting to be ready)</p>
</li>
</ul>
<p>As you can see now I have 2 pods with the same transactional-id.</p>
<p>As far as I understood, these IDs have to be the same across restarts and unique.</p>
<p>How can I implement something that gives me consistent IDs? Is there someone that have dealt with this problem?</p>
<p>The problem with these IDs are only for the Kubernetes Deployments, not for the Stateful-Sets, as they have a stable identifier as name. I don't want to convert all deployment to stateful sets to solve this problem as I think it is not the correct way to handle this scenario.</p>

Justin
<p>The only way to guarantee the uniqueness of <code>Pods</code> is to use <a href="https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/" rel="nofollow noreferrer">StatefulSet</a>.
<code>StatefulSets</code> will allow you to keep the number of replicas alive but everytime pod dies it will be replaced with the same host and configuration. That will prevent data loss that is required. </p>

<p>Service in <code>Statefulset</code> must be <code>headless</code> because since each pod is going to be unique, so you are going to need certain traffic to reach certain pods. </p>

<p>Every <code>pod</code> require a <code>PVC</code> (in order to store data and recreate whenever pod is deleted from that data).</p>

<p><a href="https://blog.yugabyte.com/best-practices-for-deploying-confluent-kafka-spring-boot-distributed-sql-based-streaming-apps-on-kubernetes/" rel="nofollow noreferrer">Here</a> is a great article describing why <code>StatefulSet</code> should be used in similar case.</p>
