<p>I am exploring different strategies for handling shutdown gracefully in case of deployment/crash. I am using the Spring Boot framework and Kubernetes. In a few of the services, we have tasks that can take around 10-20 minutes(data processing, large report generation). How to handle pod termination in these cases when the task is taking more time. For queuing I am using Kafka.</p>

Shivam Singh
<blockquote>
<p>we have tasks that can take around 10-20 minutes(data processing, large report generation)</p>
</blockquote>
<p>First, this is more of a <em>Job/Task</em> rather than a <em>microservice</em>. But similar &quot;rules&quot; applies, the node where this job is executing might terminate for upgrade or other reason, so your <em>Job/Task</em> must be <em>idempotent</em> and be able to be <em>re-run</em> if it crashes or is terminated.</p>
<blockquote>
<p>How to handle pod termination in these cases when the task is taking more time. For queuing I am using Kafka.</p>
</blockquote>
<p>Kafka is a good technology for this, because it is able to let the client Jon/Task to be <em>idempotent</em>. The job receives the data to process, and after processing it can &quot;commit&quot; that it has processed the data. If the Task/Job is terminated before it has processed the data, a new Task/Job will spawn and continue processing on the &quot;offset&quot; that is not yet committed.</p>
