<p>Our cloud application consists of 3 tightly coupled Docker containers, Nginx, Web and Mongo. Currently we run these containers on a single machine. However as our users are increasing we are looking for a solution to scale. Using Kubernetes we would form a multi container pod. If we are to replicate we need to replicate all 3 containers as a unit. Our cloud application is consumed by mobile app users. Our app can only handle approx 30000 users per Worker node and we intend to place a single pod on a single worker node. Once a mobile device is connected to worker node it must continue to only use that machine ( unique IP address )</p>

<p>We plan on using Kubernetes to manage the containers. Load balancing doesn't work for our use case as a mobile device needs to be tied to a single machine once assigned and each Pod works independently with its own persistent volume.  However we need a way of spinning up new Pods on worker nodes if the number of users goes over 30000 and so on. </p>

<p>The idea is we have some sort of custom scheduler which assigns a mobile device a Worker Node ( domain/ IPaddress) depending on the number of users on that node.</p>

<p>Is Kubernetes a good fit for this design and how could we implement a custom pod scale algorithm. </p>

<p>Thanks</p>

tech74
<p>Piggy-Backing on the answer of Jonah Benton:</p>

<p>While this is technically possible - your problem is not with Kubernetes it's with your Application! Let me point you the problem:</p>

<blockquote>
  <p>Our cloud application consists of 3 tightly coupled Docker containers, Nginx, Web, and Mongo.</p>
</blockquote>

<p>Here is your first problem: Is you can only deploy these three containers together and not independently - you cannot scale one or the other!
While MongoDB can be scaled to insane loads - if it's bundled with your web server and web application it won't be able to...</p>

<p>So the first step for you is to break up these three components so they can be managed independently of each other. Next:</p>

<blockquote>
  <p>Currently we run these containers on a single machine.</p>
</blockquote>

<p>While not strictly a problem - I have serious doubt's what it would mean to scale your application and what the challenges that come with scalability!</p>

<blockquote>
  <p>Once a mobile device is connected to worker node it must continue to only use that machine ( unique IP address )</p>
</blockquote>

<p>Now, this IS a problem. You're looking to run an application on Kubernetes but I do not think you understand the consequences of doing that: Kubernetes orchestrates your resources. This means it will move pods (by killing and recreating) between nodes (and if necessary to the same node). It does this fully autonomous (which is awesome and gives you a good night sleep) If you're relying on clients sticking to a single nodes IP, you're going to get up in the middle of the night because Kubernetes tried to correct for a node failure and moved your pod which is now gone and your users can't connect anymore. You need to leverage the load-balancing features (services) in Kubernetes. Only they are able to handle the dynamic changes that happen in Kubernetes clusters.</p>

<blockquote>
  <p>Using Kubernetes we would form a multi container pod.</p>
</blockquote>

<p>And we have another winner - No! You're trying to treat Kubernetes as if it were your on-premise infrastructure! If you keep doing so you're going to fail and curse Kubernetes in the process!</p>

<p>Now that I told you some of the things you're thinking wrong - what a person would I be if I did not offer some advice on how to make this work:</p>

<p>In Kubernetes your three applications should not run in one pod! They should run in separate pods:</p>

<ul>
<li>your webservers work should be done by <a href="https://kubernetes.io/docs/concepts/services-networking/ingress/" rel="nofollow noreferrer">Ingress</a> and since you're already familiar with nginx, <a href="https://github.com/kubernetes/ingress-nginx" rel="nofollow noreferrer">this is probably the ingress you are looking for!</a></li>
<li>Your web application should be a simple <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/" rel="nofollow noreferrer">Deployment</a> and be exposed to ingress through a <a href="https://kubernetes.io/docs/concepts/services-networking/service/" rel="nofollow noreferrer">Service</a></li>
<li>your database should be a separate deployment which you can either do manually <a href="https://kubernetes.io/blog/2017/01/running-mongodb-on-kubernetes-with-statefulsets" rel="nofollow noreferrer">through a statefullset</a> or <a href="https://www.kubestack.com/catalog/mongodb" rel="nofollow noreferrer">(more advanced) through an operator</a> and also exposed to the web application trough a Service</li>
</ul>

<p>Feel free to ask if you have any more questions!</p>
