<p>I have a classic microservice architecture. So, there are differ applications. Each application may have <code>1..N</code> instances. The system is deployed to <code>Kubernetes.</code> So, we have many differ <code>PODs</code>, which can start and stop in any time.</p>

<p>I want to implement <a href="https://www.confluent.io/blog/transactions-apache-kafka/" rel="nofollow noreferrer">read-process-write</a> pattern, so I need Kafka transactions. </p>

<p>To configure transactions, I need to set some <code>transaction id</code> for each Kafka producer. (Actually, I need <code>transaction-id-prefix</code>, because of I use Spring for my applications, and it has such <code>API</code>). These <code>IDs</code> have to be the same, after application is restarted.</p>

<p>So, how to choose Kafka transaction id for several applications, hosted in Kubernetes?</p>

Denis
<p>If the consumer starts the transaction (read-process-write) then the transaction id prefix must be the same for all instances of the same app (so that zombie fencing works correctly after a rebalance). The actual transaction id used is <code>&lt;prefix&gt;&lt;group&gt;.&lt;topic&gt;.&lt;partition&gt;</code>.</p>
<p>If you have multiple apps, they should have unique prefixes (although if they consume from different topics, they will be unique anyway).</p>
<p>For producer-only transactions, the prefix must be unique in each instance (to prevent kafka fencing the producers).</p>
<p><strong>EDIT</strong></p>
<p>Note that <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-447%3A+Producer+scalability+for+exactly+once+semantics" rel="nofollow noreferrer">KIP-447</a> changed all this; it is no longer necessary (when using <code>EOSMode.V2</code> aka <code>BETA</code>) to keep the tranactional id the same - consumer metadata is used for fencing instead.</p>
