<p>The image below show the two pipelines that we have in my company to manage the life cycle of a model.</p>
<p><a href="https://i.stack.imgur.com/IDY5L.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/IDY5L.png" alt="enter image description here" /></a></p>
<ul>
<li>The first pipeline, &quot;Application&quot;, relates to the creation of the application component that host the model and has the inference logic.</li>
<li>The second, that of &quot;Model&quot;, is a pipeline that leads to the generation of the model in binary format.</li>
</ul>
<p>Together (model and application) will be deployed in our orchestrator (a kubernetes cluster).</p>
<p><a href="https://i.stack.imgur.com/dO1LB.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/dO1LB.png" alt="enter image description here" /></a></p>
<p>I am in a situation where the application logic does not change but the models do.
I could find myself in the situation below.</p>
<p><a href="https://i.stack.imgur.com/tKSeF.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/tKSeF.png" alt="enter image description here" /></a></p>
<p>I suppose there are two approach to manage the runtime model refresh on the orchestrator (hope someone suggests me other possibilities that I haven't thought about):</p>
<ol>
<li>In the application logic; The code manages the refresh through a thread by taking the new model.
<ul>
<li><em>Pros</em>: A new container is not generated</li>
<li><em>Cons</em>: Ability to introduce a bug.</li>
</ul>
</li>
<li>Through the pipeline; The pipeline must be triggered by an event (in my case a merge on a git branch) and bring the container by performing a rolling update. The new container at run will load the new model.
<ul>
<li><em>Pros</em>: Existing process</li>
<li><em>Cons</em>: Each new version of the model should provide for a new build of the container, even if the application logic has not changed.</li>
</ul>
</li>
</ol>
<p>[<strong>Question</strong>]
Are there any best practices for these cases (perhaps through a system of tags on the images) that someone can suggest me?</p>
<p>Thank you
Kipliko</p>

kipliko
<p>The most seamless way is to do a <a href="https://kubernetes.io/docs/tutorials/kubernetes-basics/update/update-intro/" rel="nofollow noreferrer">rolling update</a> via k8s/kubectl. This will require a new container, however, this is considered a best practice, as each container stays atomic and reproducible. Updating the model via threads would be difficult to debug.</p>
<p>Another scenario you could do is <a href="https://semaphoreci.com/blog/continuous-blue-green-deployments-with-kubernetes" rel="nofollow noreferrer">blue-green deployment</a> using Istio, and slowly move traffic between the old and new model, although this would require a bit more overhead.</p>
