<p>I've encountered a rather rookie issue with k8s. I'm particularly new to k8s, and setup staging and production services / deployments for a Django - celery - redis application within a cluster. However. In my excitement that I actually managed to get something working, I didn't check to think if it was 100% correct.</p>
<p>Essentially, I've noticed that the pre-production Django application doesn't care which celery deployment it references when dispatching a periodic task. It might go to staging, it might try the pre-production deployment. <strong>THIS IS BAD</strong>.</p>
<p>So, I've been looking at labels and selectors, as well as namespaces.</p>
<p>However, I should probably slow down - my first question, how would I use something native to k8s to run different environments of deployments, such that they are all isolated from each other. So the pre-production Django application can only talk to the pre-production celery-worker or pre-production celery-beat deployments...</p>
<p>*My answer I feel is to use labels and selectors? But ... is it best to use namespaces?</p>
<p>Any pro-guidance around this subject would be amazing.</p>

Micheal J. Roberts
<p>Aside from creating a new cluster per environment, you can separate deployments by namespace or just different stacks in one namespace. The last case (the one you use now) is the easiest to shoot in the leg since you have to change a lot of thing to make it isolated. At very least you need a different set of resource names (both in manifests and configuration) and labels to match.</p>
<p>Out of the three methods I think namespace separation is the easiest; it works on DNS-based service discovery. Suppose you have a copy of redis and your application in two different namespaces (<code>dev</code> and <code>prod</code> for example). Both instances of the app are configured to use redis at <code>redis:6379</code>. When they call DNS to resolve the hostname, <code>CoreDNS</code> (the internal DNS service) would respond with different answers depending on which namespace the request came from. And so your app in <code>dev</code> namespace will get an IP-address of redis in <code>dev</code> namespace, and the app from <code>prod</code> namespace will contact redis from <code>prod</code> namespace. This method does not apply any restriction, if you wish you can specifically make it so that both apps use the same copy of redis. For that, instead of <code>redis:6379</code> you have to use a full service DNS name, like this:</p>
<p><code>redis.&lt;namespace&gt;.svc.cluster.local:6379</code></p>
<p>Regardless of whatever method you choose to go with, I strongly recommend you to get familiar with <a href="https://kustomize.io/" rel="nofollow noreferrer">Kustomize</a>, <a href="https://helm.sh/" rel="nofollow noreferrer">Helm</a>, or both. These tools are to help you avoid duplicating resource manifests and thus spend less time spawning and managing instances. I will give you a minimal example for <code>Kustomize</code> because it is built in <code>kubectl</code>. Consider the following directory structure:</p>
<pre><code>.
├── bases
│   └── my-app
│       ├── deployment.yml # your normal deployment manifest
│       └── kustomization.yml
└── instances
    ├── prod
    │   ├── kustomization.yml
    │   └── namespace.yml # a manifest that creates 'prod' namespace
    └── test
        ├── kustomization.yml
        └── namespace.yml # a manifest that creates 'test' namespace
</code></pre>
<p><code>bases</code> is where you keep a non-specific skeleton of your application. This isn't meant to be deployed, like a class it has to be instantiated. <code>instances</code> is where you describe various instances of your application. Instances are meant to be deployed.</p>
<p><strong>bases/my-app/kustomization.yml</strong>:</p>
<pre class="lang-yaml prettyprint-override"><code># which manifests to pick up
resources:
  - deployment.yml
</code></pre>
<p><strong>instances/prod/kustomization.yml</strong>:</p>
<pre class="lang-yaml prettyprint-override"><code># refer what we deploy
bases:
  - ../../bases/my-app
resources:
  - namespace.yml
# and this overrides namespace attribute for all manifests referenced above
namespace: prod
</code></pre>
<p><strong>instances/test/kustomization.yml</strong>:</p>
<pre class="lang-yaml prettyprint-override"><code># the same as above, only the namespace is different
bases:
  - ../../bases/my-app
resources:
  - namespace.yml
namespace: test
</code></pre>
<p>Now if you go into <code>instances</code> directory and use <code>kubectl apply -k prod</code> you will deploy <code>deployment.yml</code> to <code>prod</code> namespace. Similarly <code>kubectl apply -k test</code> will deploy it to the <code>test</code> namespace.</p>
<p>And this is how you can create several identical copies of your application stack in different namespaces. It should be fairly isolated unless some shared resources from other namespaces involved. In other words, if you deploy each component (such as the database) per namespace and those components are not configured to access components from other namespaces - it will work as expected.</p>
<p>I encourage you to read more on <a href="https://kustomize.io/" rel="nofollow noreferrer">Kustomize</a> and <a href="https://helm.sh/" rel="nofollow noreferrer">Helm</a>, since namespace overriding is just a basic thing these can do. You can manage labels, configuration, names, stack components and more.</p>
