<p>My goal is to do a zero downtime updates on kubernetes.</p>
<p>But, there is a problem related to a file upload.</p>
<p>the situation is when user uploads a file, webserver stores it first. and WAS saves metadatas of a file to DB.</p>
<p>so the problem is when we updates webservers. webserver are not gonna wait for the request to be finished. and the file uploads/downloads services are gonna be failed(if clients are conneted to a webserver which is gonna shut down).</p>
<p>what am i supposed to do about this?</p>

ABCD1133
<p>In short, there is no magic tool in Kubernetes that can solve it for any type of application.</p>
<p>What is the main goal?</p>
<ul>
<li><p>Delete a pod (if you can delete a pod gracefully that is big step)</p>
</li>
<li><p>App supports both versions at the same time (for roll out and roll back)</p>
</li>
</ul>
<p>So how to achieve zero downtime deployments and updates?</p>
<p>Kubernetes/Docker:</p>
<ul>
<li><p>Application is running as special PID 1 so it can receive <code>SIGTERM</code> (standard graceful shutdown) signal directly</p>
</li>
<li><p>You specify <code>terminationGracePeriodSeconds</code> in <code>StatefulSet</code> or <code>Deployment</code>. When you scale down an application (or delete a pod to replace with new pod), no new connections are routed, it sends <code>SIGTERM</code> to app and waits for <code>terminationGracePeriodSeconds</code> time. Usually it's up to 5-10 minutes to drain connections, but could be even hours to finish long ones. If app understands <code>SIGTERM</code> as you wanted it can finish this earlier.</p>
</li>
<li><p>Just working <code>readinessProbe</code> check</p>
</li>
</ul>
<p>Application:</p>
<ul>
<li><p>Ideally understands <code>SIGTERM</code> and closes operations gracefully</p>
</li>
<li><p>App should be able to retry connection or operation if something failed in first attempt (e.g. API call from frontend to backend, DB query from backend, etc.) - this helps to retry operation on new pod and in general retry is a good thing in a highly available systems</p>
</li>
<li><p>App does its job in smaller chunks and with mentioned retry needs, for example using <a href="http://resumablejs.com" rel="nofollow noreferrer">http://resumablejs.com</a> to avoid long file transfers - long connections</p>
</li>
<li><p>Strategy for schema changes, both versions should be supported at the same time (so for instance if you add new column to DB, it's better to add it in first release, then use it in second release and so many other techniques)</p>
</li>
</ul>
<p>Application (last resort):</p>
<ul>
<li>Some companies which cannot afford downtime, but it's to complicated/not possible to deploy a new version decide to queue new connections (additional code) while app is upgrading. So files and DB records are imported from old into new version to finish zero downtime deployment.</li>
</ul>
