<p>I have created 2 tenants(tenant1,tenant2) in 2 namespaces tenant1-namespace,tenant2-namespace</p>

<p>Each tenant has db pod and its services</p>

<p>How to isolate db pods/service i.e. how to restrict pod/service from his namespace to access other tenants db pods ?</p>

<p>I have used service account for each tenant and applied network policies so that namespaces are isolated. </p>

<pre><code>kubectl get svc --all-namespaces

tenant1-namespace   grafana-app            LoadBalancer   10.64.7.233    104.x.x.x   3000:31271/TCP   92m
tenant1-namespace   postgres-app           NodePort       10.64.2.80     &lt;none&gt;      5432:31679/TCP   92m
tenant2-namespace   grafana-app            LoadBalancer   10.64.14.38    35.x.x.x    3000:32226/TCP   92m
tenant2-namespace   postgres-app           NodePort       10.64.2.143    &lt;none&gt;      5432:31912/TCP   92m
</code></pre>

<p>So</p>

<p>I want to restrict grafana-app to use only his postgres db in his namespace only, not in other namespace.</p>

<p>But problem is that using DNS qualified service name (<code>app-name.namespace-name.svc.cluster.local</code>) 
its allowing to access each other db pods (grafana-app in namespace tenant1-namespace can have access to postgres db in other tenant2-namespace via <code>postgres-app.tenant2-namespace.svc.cluster.local</code></p>

<p>Updates : network policies</p>

<p>1) </p>

<pre><code>kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: deny-from-other-namespaces
spec:
  podSelector:
    matchLabels:
  ingress:
  - from:
    - podSelector: {}
</code></pre>

<p>2) </p>

<pre><code>kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: web-allow-external
spec:
  podSelector:
    matchLabels:
      app: grafana-app
  ingress:
  - from: []
</code></pre>

<ul>
<li><p><strong>Your <code>NetworkPolicy</code> objects are correct</strong>, I created an example with them and will demonstrate bellow.</p></li>
<li><p><strong>If you still have access</strong> to the service <strong>on the other namespace</strong> using FQDN, <strong>your <code>NetworkPolicy</code> may not be fully enabled</strong> on your cluster.</p></li>
</ul>

<p>Run <code>gcloud container clusters describe "CLUSTER_NAME" --zone "ZONE"</code> and look for these two snippets:</p>

<ul>
<li>At the beggining of the description it shows if the NetworkPolicy Plugin is enabled <strong>at Master level</strong>, it should be like this:</li>
</ul>

<pre><code>addonsConfig:
networkPolicyConfig: {}
</code></pre>

<ul>
<li>At the middle of the description, you can find if the NetworkPolicy is <strong>enabled on the nodes</strong>. It should look like this:</li>
</ul>

<pre><code>name: cluster-1
network: default
networkConfig:
  network: projects/myproject/global/networks/default
  subnetwork: projects/myproject/regions/us-central1/subnetworks/default
networkPolicy:
  enabled: true
  provider: CALICO
</code></pre>

<ul>
<li>If any of the above is different, check here: <a href="https://cloud.google.com/kubernetes-engine/docs/how-to/network-policy#gcloud" rel="nofollow noreferrer">How to Enable Network Policy in GKE</a></li>
</ul>

<hr>

<p><strong>Reproduction:</strong></p>

<ul>
<li>I'll create a simple example, I'll use <code>gcr.io/google-samples/hello-app:1.0</code> image for tenant1 and <code>gcr.io/google-samples/hello-app:2.0</code> for tenant2, so it's simplier to see where it's connecting but i'll use the names of your environment:</li>
</ul>

<pre><code>$ kubectl create namespace tenant1
namespace/tenant1 created
$ kubectl create namespace tenant2
namespace/tenant2 created

$ kubectl run -n tenant1 grafana-app --generator=run-pod/v1 --image=gcr.io/google-samples/hello-app:1.0 
pod/grafana-app created
$ kubectl run -n tenant1 postgres-app --generator=run-pod/v1 --image=gcr.io/google-samples/hello-app:1.0 
pod/postgres-app created

$ kubectl run -n tenant2 grafana-app --generator=run-pod/v1 --image=gcr.io/google-samples/hello-app:2.0 
pod/grafana-app created
$ kubectl run -n tenant2 postgres-app --generator=run-pod/v1 --image=gcr.io/google-samples/hello-app:2.0 
pod/postgres-app created

$ kubectl expose pod -n tenant1 grafana-app --port=8080 --type=LoadBalancer
service/grafana-app exposed
$ kubectl expose pod -n tenant1 postgres-app --port=8080 --type=NodePort
service/postgres-app exposed

$ kubectl expose pod -n tenant2 grafana-app --port=8080 --type=LoadBalancer
service/grafana-app exposed
$ kubectl expose pod -n tenant2 postgres-app --port=8080 --type=NodePort
service/postgres-app exposed

$ kubectl get all -o wide -n tenant1
NAME               READY   STATUS    RESTARTS   AGE    IP          NODE                                         
pod/grafana-app    1/1     Running   0          100m   10.48.2.4   gke-cluster-114-default-pool-e5df7e35-ez7s
pod/postgres-app   1/1     Running   0          100m   10.48.0.6   gke-cluster-114-default-pool-e5df7e35-c68o

NAME                   TYPE           CLUSTER-IP   EXTERNAL-IP     PORT(S)          AGE   SELECTOR
service/grafana-app    LoadBalancer   10.1.23.39   34.72.118.149   8080:31604/TCP   77m   run=grafana-app
service/postgres-app   NodePort       10.1.20.92   &lt;none&gt;          8080:31033/TCP   77m   run=postgres-app

$ kubectl get all -o wide -n tenant2
NAME               READY   STATUS    RESTARTS   AGE    IP          NODE                                         
pod/grafana-app    1/1     Running   0          76m    10.48.4.8   gke-cluster-114-default-pool-e5df7e35-ol8n
pod/postgres-app   1/1     Running   0          100m   10.48.4.5   gke-cluster-114-default-pool-e5df7e35-ol8n

NAME                   TYPE           CLUSTER-IP    EXTERNAL-IP      PORT(S)          AGE   SELECTOR
service/grafana-app    LoadBalancer   10.1.17.50    104.154.135.69   8080:30534/TCP   76m   run=grafana-app
service/postgres-app   NodePort       10.1.29.215   &lt;none&gt;           8080:31667/TCP   77m   run=postgres-app
</code></pre>

<ul>
<li>Now, let's deploy your two rules:  The first blocking all traffic from outside the namespace, the second allowing ingress the <code>grafana-app</code> from outside of the namespace:</li>
</ul>

<pre><code>$ cat default-deny-other-ns.yaml 
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: deny-from-other-namespaces
spec:
  podSelector:
    matchLabels:
  ingress:
  - from:
    - podSelector: {}

$ cat allow-grafana-ingress.yaml 
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: web-allow-external
spec:
  podSelector:
    matchLabels:
      run: grafana-app
  ingress:
  - from: []
</code></pre>

<ul>
<li>Let's review the rules for <a href="https://kubernetes.io/docs/concepts/services-networking/network-policies/#isolated-and-non-isolated-pods" rel="nofollow noreferrer">Network Policy Isolation</a>:</li>
</ul>

<blockquote>
  <p>By default, pods are non-isolated; they accept traffic from any source.</p>
  
  <p>Pods become isolated by having a NetworkPolicy that selects them. Once there is any NetworkPolicy in a namespace selecting a particular pod, that pod will reject any connections that are not allowed by any NetworkPolicy. (Other pods in the namespace that are not selected by any NetworkPolicy will continue to accept all traffic.)</p>
  
  <p><strong>Network policies do not conflict; they are additive</strong>. If any policy or policies select a pod, the pod is restricted to what is allowed <strong>by the union of those policies</strong>' ingress/egress rules. Thus, order of evaluation does not affect the policy result.</p>
</blockquote>

<ul>
<li>Then we will apply the rules on both namespaces because the scope of the rule is the namespace it's assigned to:</li>
</ul>

<pre><code>$ kubectl apply -n tenant1 -f default-deny-other-ns.yaml 
networkpolicy.networking.k8s.io/deny-from-other-namespaces created
$ kubectl apply -n tenant2 -f default-deny-other-ns.yaml 
networkpolicy.networking.k8s.io/deny-from-other-namespaces created

$ kubectl apply -n tenant1 -f allow-grafana-ingress.yaml 
networkpolicy.networking.k8s.io/web-allow-external created
$ kubectl apply -n tenant2 -f allow-grafana-ingress.yaml 
networkpolicy.networking.k8s.io/web-allow-external created
</code></pre>

<ul>
<li>Now for final testing, I'll log inside <code>grafana-app</code> in <code>tenant1</code> and try to reach the <code>postgres-app</code> in both namespaces and check the output:</li>
</ul>

<pre><code>$ kubectl exec -n tenant1 -it grafana-app -- /bin/sh
/ ### POSTGRES SAME NAMESPACE ###
/ # wget -O- postgres-app:8080
Connecting to postgres-app:8080 (10.1.20.92:8080)
Hello, world!
Version: 1.0.0
Hostname: postgres-app

/ ### GRAFANA OTHER NAMESPACE ###
/ # wget -O- --timeout=1 http://grafana-app.tenant2.svc.cluster.local:8080
Connecting to grafana-app.tenant2.svc.cluster.local:8080 (10.1.17.50:8080)
Hello, world!
Version: 2.0.0
Hostname: grafana-app

/ ### POSTGRES OTHER NAMESPACE ###
/ # wget -O- --timeout=1 http://postgres-app.tenant2.svc.cluster.local:8080
Connecting to postgres-app.tenant2.svc.cluster.local:8080 (10.1.29.215:8080)
wget: download timed out
</code></pre>

<ul>
<li>You can see that the DNS is resolved, but the networkpolicy blocks the access to the backend pods.</li>
</ul>

<p>If after double checking NetworkPolicy is enabled on Master and Nodes you still face the same issue let me know in the comments and we can dig further.</p>
