<p>I am <a href="https://piotrminkowski.com/2021/02/18/blue-green-deployment-with-a-database-on-kubernetes/" rel="nofollow noreferrer">reading about blue green deployment with database changes on Kubernetes.</a>  It explains very clearly and in detail how the process works:</p>
<ol>
<li>deploy new containers with the new versions while still directing traffic to the old containers</li>
<li>migrate database changes and have the services point to the new database</li>
<li>redirect traffic to the new containers and remove the old containers when there are no issues</li>
</ol>
<p>I have some questions particularly about the moment we switch from the old database to the new one.</p>
<p>In step 3 of the article, we have <code>person-v1</code> and <code>person-v2</code> services that both still point to the unmodified version of the database (postgres v1):</p>
<p><a href="https://i.stack.imgur.com/pCvg1.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/pCvg1.png" alt="before database migration" /></a></p>
<p>From this picture, having <code>person-v2</code> point to the database is probably needed to establish a TCP connection, but it would likely fail due to incompatibility between the code and DB schema. But since all incoming traffic is still directed to <code>person-v1</code> this is not a problem.</p>
<p>We now modify the database (to postgres v2) and switch the traffic to <code>person-v2</code> (step 4 in the article).  <strong>I assume that both the DB migration and traffic switch happen at the same time?</strong>  That means it is impossible for <code>person-v1</code> to communicate with postgres v2 or <code>person-v2</code> to communicate with postgres v1 at any point during this transition?  Because this can obviously cause errors (i.e. inserting data in a column that doesn't exist yet/anymore).</p>
<p><a href="https://i.stack.imgur.com/PcJO5.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/PcJO5.png" alt="after database migration" /></a></p>
<p>If the above assumption is correct, then <strong>what happens if during the DB migration new data is inserted in postgres v1</strong>?  Is it possible for data to become lost with unlucky timing?  Just because the traffic switch happens at the same time as the DB switch, does not mean that any ongoing processes in <code>person-v1</code> can not still execute DB statements.  It would seem to me that any new inserts/deletes/updates would need to propagate to postgres v2 as well for as long as the migration is still in progress.</p>

<p>Even when doing blue-green for the application servers, you still have to follow normal rules of DB schema compatibility. All schema changes need to be backwards compatible for whatever you consider one full release cycle to be. Both services talk to the same DB during the switchover time but thanks to careful planning each can understand the data from the other and all is well.</p>
