<p>I have found several people asking about how to start a container running a DB, then run a different container that runs maintenance/migration on the DB which then exits. Here are all of the solutions I've examined and what I think are the problems with each:</p>
<ol>
<li><a href="https://kubernetes.io/docs/concepts/workloads/pods/init-containers/" rel="nofollow noreferrer">Init Containers</a> - This wont work because these run before the main container is up and they block the starting of the main container until they successfully complete.</li>
<li><a href="https://kubernetes.io/docs/tasks/configure-pod-container/attach-handler-lifecycle-event/" rel="nofollow noreferrer">Post Start Hook</a> - If the <em>postStart</em> hook could start containers rather than simply exec a command inside the container then this would work. Unfortunately, the container with the DB does not (and should not) contain the rather large maintenance application required to run it this way. This would be a violation of the principle that each component should <em>do one thing and do it well</em>.</li>
<li><a href="https://medium.com/bb-tutorials-and-thoughts/kubernetes-learn-sidecar-container-pattern-6d8c21f873d" rel="nofollow noreferrer">Sidecar Pattern</a> - This WOULD work if the <em>restartPolicy</em> were assignable or overridable at the container level rather than the pod level. In my case the maintenance container should terminate successfully before the pod is considered <em>Running</em> (just like would be the case if the <em>postStart</em> hook could run a container) while the DB container should <em>Always</em> restart.</li>
<li>Separate Pod - Running the maintenance as a separate pod can work, but the DB shouldn't be considered up until the maintenance runs. That means managing the <em>Running</em> state has to be done completely independently of Kubernetes. Every other container/pod in the system will have to do a custom check that the maintenance has run rather than a simple check that the DB is up.</li>
<li>Using a <a href="https://kubernetes.io/docs/concepts/workloads/controllers/job/" rel="nofollow noreferrer">Job</a> - Unless I misunderstand how these work, this would be equivalent to the above (&quot;Separate Pod&quot;).</li>
<li><em>OnFailure</em> restart policy with a Sidecar - This means using a <em>restartPolicy</em> of <em>OnFailure</em> for the POD but then hacking the DB container so that it always exits with an error. This is doable but obviously just a hacked workaround. EDIT: This also causes problems with the state of the POD. When the maintenance runs and stays up and both containers are running, the state of the POD is <em>Ready</em>, but once the maintenance container exits, even with a SUCCESS (0 exit code), the state of the POD goes to <em>NotReady 1/2</em>.</li>
</ol>
<p>Is there an option I've overlooked or something I'm missing about the above solutions? Thanks.</p>

<p>One option would be to use the <code>Sidecar pattern</code> with 2 slight changes to the approach you described:</p>
<ol>
<li>after the maintenance command is executed, you keep the container running with a <code>while : ; do sleep 86400; done</code> command or something similar.</li>
<li>You set an appropriate <code>startupProbe</code> in place that resolves successfully only when your maintenance command is executed successfully. You could for example create a file <code>/maintenance-done</code> and use a startupProbe like this:</li>
</ol>
<pre><code>startupProbe:
  exec:
    command:
    - cat
    - /maintenance-done
  initialDelaySeconds: 5
  periodSeconds: 5
</code></pre>
<p>With this approach you have the following outcome:</p>
<ol>
<li>Having the same <code>restartPolicy</code> for both your database and sidecar containers works fine thanks to the <code>sleep hack</code>.</li>
<li>You Pod only becomes ready when both containers are ready. In the sidecar container case this happens when the startupProbe succeedes.</li>
</ol>
<p>Furthermore, there will be no noticeable overhead in your pod: even if the sidecar container keeps running, it will consume close to zero resources since it is only running the sleep command.</p>
