<p>We have a Kubernetes deployment consisting of a nodejs front end and an nginx backend. We're finding that the two deployments work fine in Kubernetes individually, but when they are both deployed requests to the front end return a 404 almost exactly 50% of the time.</p>
<p>It's natural to assume there is an issue with our virtual service, but this seems to not be the case, based on the fact that the deployment of the vs/gateway is not sufficient to cause the issue. It also seems that if we deploy a different, unrelated image in the backend, the front-end continues to work without 404 errors.</p>
<p>The app was originally generated via JHipster, and we manually separated the front-end and backend components. The front-end is nodejs, the backend is Java/nginx. The app works locally, but fails in a k8s deployment.</p>
<p>Also, our Kubernetes deployment is in Rancher.</p>
<p>Experiments seem to indicate it is related to something in our back-end deployment, so I'm including our backend deployement.yaml below:</p>
<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: ourapp-be-custom-mount
spec:
  revisionHistoryLimit: 3
  replicas: 1
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 25%
  template:
    spec:
      containers:
        - name: ourapp-be-custom-mount
          image: &quot;IMAGE_SET_BY_OVERLAYS_KUSTOMIZATION&quot;
          envFrom:
            - configMapRef:
                name: ourapp-be-config
          ports:
          - name: http
            containerPort: 8080
          resources:
            limits:
              cpu: &quot;0.5&quot;
              memory: &quot;2048Mi&quot;
            requests:
              cpu: &quot;0.1&quot;
              memory: &quot;64Mi&quot;
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - mountPath: /usr/share/h2/data
              name: ourapp-db-vol01-custom-mount

          securityContext:
            runAsNonRoot: true
            runAsUser: 1000
      imagePullSecrets:
        - name: regcred-nexus
      volumes:
      - name: ourapp-db-vol01-custom-mount
        persistentVolumeClaim:
          claimName: ourapp-db-pvc-volume01-custom-mount
      terminationGracePeriodSeconds: 30
</code></pre>

<p>Each service needs to point to a different app. You can verify in Rancher that each service points to a different app. Check your yaml. If using Kustomize, the commonLabels:app can trip you up. Make sure it points to different apps for frontend and backend.</p>
