<p>I have two services name Product and  Order.
Order table inside OrderDb has price and productId columns for storing product price and product id that ordered. Order services has 3 replica.</p>
<p>Now, suppose a product is ordered and it's id 80, and a series of sequential update event fired from product service to order services for that particular product:</p>
<pre><code>event1:{productId: 80, price: 500}
event2:{productId: 80, price: 600}
event3:{productId: 80, price: 400}
event4:{productId: 80, price: 900}
event5:{productId: 80, price: 100}
</code></pre>
<p>so the end price should be 100 for that product, but sometimes these events are processed in random order such as</p>
<pre><code>event1:{productId: 80, price: 500}
event2:{productId: 80, price: 600}
event5:{productId: 80, price: 100}
event4:{productId: 80, price: 900}
event3:{productId: 80, price: 400}
</code></pre>
<p>since event 3 processed last, price become 400.</p>

Rafiq
<p>This is generally up to your database. I see you put NATS in the tags so I'm assuming you mean you have some kind of worker queue model, but you probably have a database of record behind that with its own consistency model. With event streaming systems where you want to defend against out of order or multi-delivery, you would include more info in the queue message such as an object version, or just the previous price. In the latter, simpler case it would be like</p>
<pre><code>event1:{productId: 80, price: 500, oldPrice: 0}
event2:{productId: 80, price: 600, oldPrice: 500}
event3:{productId: 80, price: 400, oldPrice: 600}
event4:{productId: 80, price: 900, oldPrice: 400}
event5:{productId: 80, price: 100, oldPrice: 900}
</code></pre>
<p>That would let your code refuse to apply the operation if the base state no longer matches. But that's pretty limiting, you don't want everything to fail after a re-order, you just want convergent behavior. This the point where I just yell &quot;VECTOR CLOCKS&quot; and leap out of the window. Designing distributed, convergent systems is indeed quite hard, look up the term CRDT as a starting point.</p>
