<p>Let's say I have an system running on Kubernetes using rolling updates or AWS ECS using blue/green deployments or any other solution which offers zero-downtime deployments. The key point here is that new and existing version can coexist and they use the same relational database.</p>
<p>We have a table in the database called <code>users</code> with a field <code>username</code> which we want to split in the new version into two separate fields: <code>firstname</code> and <code>lastname</code>. To do that, we change the service code accordingly and do migration of existing records. This scenario also caters for potential rollbacks. This scenario should also cater for potential rollbacks.</p>
<p>During the deployment we first run the migration then run the rolling update. As far as I understand, this can lead to situation, where the migration is done, but the service with the previous versions is still consuming traffic. That's why we do the migration in a way, that it supports working both with the current version and the new version. We add new fields, but not removing the old one yet. <code>username</code> field will be removed in a later release.</p>
<p>After deployment is finished, old version gets disabled and only new version is running, saving <code>firstname</code> and <code>lastname</code> separatelly. But here we end up in a situation, where if during the transition time a record was stored by the previous version, it will be stored in <code>username</code> field. These records are not subject of the db migraiton, because they were created after in was launched.</p>
<p>How to overcome that situation? Should the db migration (only the data migration part) be launched after the deployment or maybe there is a gap in the process described?</p>

znaczek5
<p>This is actually a good question and one that needs to be architected into the manner in which you do the upgrade.</p>
<p>Most commonly this is worked around by locking out clients during the upgrade. One way is like this:</p>
<ol>
<li>Change the password of the DB user that the clients use, or use the maintenance mode if the client supports it (e.g. if it's a website they may have a mode that displays &quot;This site is under temporary maintenance please try again later). Other sites just allow the site to go to a 5xx error during the upgrade.</li>
<li>Perform the upgrade. Since the clients can no longer access it, you won't have any chance of a legit update</li>
<li>Wait for the rollout to complete</li>
<li>Test the new version (use the new credentials here)</li>
<li>Change the password back to the original password</li>
<li>Check you can still authenticate using the original credentials</li>
<li>Remove the maintenance mode if enabled</li>
<li>Check the clients are able to connect and monitor for any errors</li>
</ol>
