<p>I'm still new to Kubernetes so please excuse if this is a silly question.</p>

<p>I'm architecting a system which includes: </p>

<ul>
<li>an MQTT broker</li>
<li>a set of (containerized) microservices that publish and subscribe to it</li>
<li>a Redis cache that the microservices read and write to. </li>
</ul>

<p>We will certainly need multiplicity of all of these components as we scale.</p>

<p>There is a natural division in the multiplicity of each of these things: they each pertain to a set of intersections in a city. A publishing or subscribing microservice will handle 1 or more intersections. The MQTT broker instance and the Redis instance each could be set up to handle n intersections. </p>

<p>I am wondering if it makes sense to try to avoid unnecessary network hops in Kubernetes by trying to divide things up by intersection and put all containers related to a given set of intersections on one node. Would this mean putting them all on a single pod, or is there another way? </p>

<p>(By the way, there will still be other publishers and subscribers that need to access the MQTT broker that are not intersection-specific.)</p>

<p>This is more of an opinion question.</p>

<blockquote>
  <p>Would this mean putting them all on a single pod, or is there another way?</p>
</blockquote>

<p>I would certainly avoid putting them all in one Pod. In theory, you can put anything in a single pod, but the general practice is to add lightweight sidecars that handle a very specific function. </p>

<p>IMO an MQTT broker, a Redis datastore and a subscribe/publish app seem like a lot of to put in a single pod.</p>

<p>Possible Disadvantages:</p>

<ul>
<li>Harder to debug because you may not know where the failure comes from.</li>
<li>A publish/subscriber is generally more of a stateless application and MQTT &amp; Redis would stateful. <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/" rel="nofollow noreferrer">Deployments</a> are more recommended for stateless services and <a href="https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/" rel="nofollow noreferrer">StatefulSets</a> are recommended for stateful services.</li>
<li>Maybe networking latency. But you can use <a href="https://kubernetes.io/docs/concepts/configuration/assign-pod-node/" rel="nofollow noreferrer">Node Affinity and Pod Affinity</a> to mitigate that.</li>
</ul>

<p>Possible Advantages:</p>

<ul>
<li>All services sharing the same IP/Context.</li>
<li>Too much clutter in a pod.</li>
</ul>

<p>It would be cleaner if you had:</p>

<ul>
<li><a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/" rel="nofollow noreferrer">Deployment</a> for your sub/pub app.</li>
<li><a href="https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/" rel="nofollow noreferrer">StatefulSet</a> with its own storage for your Redis server.</li>
<li><a href="https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/" rel="nofollow noreferrer">Statefulset</a> with its own storage for your MQTT.</li>
</ul>

<p>Each one of these workload resources would create separate pods and you can scale independently up/down.</p>
