title: How to achieve auto scaling with AWS sage maker with single docker image and multiple model artifact?
body: <p>I am exploring AWS sage maker for deploying the machine learning models with single container/docker image and wants to achieve scalability too.</p>
  <p>For your info, we have written the <strong>training</strong> and <strong>prediction</strong> code in such a way that so, same code is being used for different customers. Only the ENV variables,VERSION_NUMBER are different according to customer.</p>
  <p><strong>VERSION_NUMBER</strong> : Refers to S3 folder name as current timestamp which contains latest model artifact.</p>
  <p>for e.g.</p>
  <pre><code>Customer name : XYZ
  docker image(training and prediction) : **intent-mapping:latest**
  s3 model atrifact path : s3_bucket/XYZ/VERSION_NUMBER/XYZ.tar.gz
  ENV variable : CUSTOMER_SUFFIX,IS_INCREMENTAL
  
  Customer name : PQR
  docker image(training and prediction) : **intent-mapping:latest**
  s3 model atrifact path : s3_bucket/PQR/VERSION_NUMBER/PQR.tar.gz
  ENV variable : CUSTOMER_SUFFIX,IS_INCREMENTAL
  </code></pre>
  <p>If you read it carefully, you will see same docker image is being refereed everywhere</p>
  <p>So,my doubts are as follow.</p>
  <ol>
  <li>Which approach should i follow to deploy the model with such a scenario?</li>
  <li>Let's say, if we use multiple model/multi container option of sage maker then how auto scaling will be managed for each customer independently?</li>
  </ol>
  <p>Please let me know your inputs on the same.</p>