<p>I'm trying to implement HorizontalPodAutoscalling on my Django API.
For this I have an Nginx-ingress redirecting request to a Service of type <code>ClusterIP</code>. There's an HPA running and sets to scale up when Memory reaches above 50%.</p>
<p>When I simulate a workload, HPA does kick in and creates news Pods. But requests still goes to the first Pod. The python process dies because of OOM in the first pod. Logs are only in the first pod too.</p>
<p>I don't understand why my Service isn't redirecting requets to new fresh pods?</p>
<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: &quot;{{ k8s.deploy.labels.app }}&quot;
    team: &quot;{{ k8s.deploy.labels.team }}&quot;
  name: &quot;{{ k8s.deploy.name }}&quot;
  namespace: &quot;{{ namespace }}&quot;
spec:
  progressDeadlineSeconds: {{ k8s.deploy.progressdeadlineseconds }}
  replicas: {{ k8s.deploy.replicas }}
  selector:
    matchLabels:
      app: &quot;{{ k8s.deploy.labels.app }}&quot;
      team: &quot;{{ k8s.deploy.labels.team }}&quot;
  template:
    metadata:
      labels:
        app: &quot;{{ k8s.deploy.labels.app }}&quot;
        team: &quot;{{ k8s.deploy.labels.team }}&quot;                                  
      containers:
      - name: &quot;{{ k8s.deploy.name }}&quot;
        image: my_image
        stdin: {{ k8s.deploy.container.stdin }}
        tty: {{ k8s.deploy.container.tty }}
        imagePullPolicy: &quot;{{ k8s.deploy.imagepullpolicy }}&quot;       
        ports:
          - containerPort: {{ k8s.deploy.container.port }}
            name: httpport       
        resources:
          limits:
            cpu: &quot;{{ k8s.deploy.containers.resources.limits.cpu }}&quot;
            memory: &quot;{{ k8s.deploy.containers.resources.limits.memory }}&quot;
          requests:
            cpu: &quot;{{ k8s.deploy.containers.resources.requests.cpu }}&quot;
            memory: &quot;{{ k8s.deploy.containers.resources.requests.memory }}&quot;
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  labels:
    app: &quot;{{ k8s.deploy.labels.app }}&quot;
    team: &quot;{{ k8s.deploy.labels.team }}&quot;
  name: &quot;{{ k8s.deploy.nginx.ingress.name }}&quot;
  namespace: &quot;{{ namespace }}&quot;
spec:
  tls:
  - hosts:
      - {{ k8s.fqdn.host }}
    secretName: &quot;{{ k8s.deploy.tls.secret.name }}&quot;
  rules:
  - host: {{ k8s.fqdn.host }}
    http:
      paths:
      - backend:
          serviceName: {{ k8s.deploy.svc.name }}
          servicePort: {{ k8s.svc.http.port }}
        path: /
        pathType: &quot;{{ k8s.ingress.http.pathtype }}&quot;
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: &quot;{{ k8s.deploy.labels.app }}&quot;
    team: &quot;{{ k8s.deploy.labels.team }}&quot;
  name: &quot;{{ k8s.deploy.svc.name }}&quot;
  namespace: &quot;{{ namespace }}&quot;
spec:
  ports:
    - name: httpport
      port: {{ k8s.svc.http.port }}
      protocol: TCP
      targetPort: {{ k8s.deploy.container.port }}
  selector:
    app: &quot;{{ k8s.deploy.labels.app }}&quot;
    team: &quot;{{ k8s.deploy.labels.team }}&quot;
  sessionAffinity: None
  type: &quot;{{ k8s.svc.type }}&quot;
</code></pre>
<p>I'm not sure if in the <code>Service</code> declaration I should use a <code>selector</code>, maybe that's the issue? The service is bouded to one pod with the <code>selector</code>?</p>
<p>At first I thought I was lacking <code>readinessprobe</code>, but I doubt it is that.
An other information, is when I launch one request to my API, the HPA is triggered (which is normal), but after 5 minutes (the default scale down of HPA I think), I get a <code>502</code> from <code>Ngnix</code>.
Also I don't get the difference between <code>ClusterIP</code> and <code>LoadBalancer</code>, I tried to user a <code>LoadBalancer</code>, but the service doesnt deploy at all.</p>
<p>Edit 1:</p>
<pre><code>kubectl describe service -n appone appone-api-test-poc-svc

Name:              appone-api-test-poc-svc
Namespace:         appone
Labels:            app=appone-api-test-poc
                   team=datalab
Annotations:       &lt;none&gt;
Selector:          app=appone-api-test-poc,team=datalab
Type:              ClusterIP
IP:                172.2.3.4
Port:              httpport  8443/TCP
TargetPort:        8443/TCP
Endpoints:         172.2.5.6:8443,172.2.5.7:8443
Session Affinity:  None
Events:            &lt;none&gt;
</code></pre>
<p>Both <code>TP</code> and <code>Endpoints</code> were changed.</p>
<p>Edit 2:</p>
<pre><code>&gt; kubectl get svc -n appone
NAME                      TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
appone-api-test-poc-svc   ClusterIP   172.2.3.4        &lt;none&gt;        8443/TCP   17h
&gt; kubectl get pods -n appone -l app=appone-api-test-poc  -o go-template='{{range .items}}{{.status.podIP}}{{&quot;\n&quot;}}{{end}}'
172.2.5.6
172.2.5.7
</code></pre>

BeGreen
<p>The file configuration from my post is correct and working.</p>
<p>The issue was that I used <code>cookies</code> with the ingress. So with postman, it kept the information of my pod and kept seending on the same pod.</p>
<p>If I would delete cookies from postman or launch a query from an other computer, it would work perfeclty.</p>
<p>This is the correct version of the <code>nginx-ingress</code> i'm using now.</p>
<pre><code>apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  labels:
    app: &quot;{{ k8s.deploy.labels.app }}&quot;
    team: &quot;{{ k8s.deploy.labels.team }}&quot;
  name: &quot;{{ k8s.deploy.nginx.ingress.name }}&quot;
  namespace: &quot;{{ namespace }}&quot;
spec:
  tls:
  - hosts:
      - {{ k8s.fqdn.host }}
    secretName: &quot;{{ k8s.deploy.tls.secret.name }}&quot;
  rules:
  - host: {{ k8s.fqdn.host }}
    http:
      paths:
      - backend:
          serviceName: {{ k8s.deploy.svc.name }}
          servicePort: {{ k8s.svc.http.port }}
        path: /
        pathType: &quot;{{ k8s.ingress.http.pathtype }}&quot;
</code></pre>
<p>This was the original configuration of <code>nginx-ingress</code> that keeps queries to the same Pod because of sticky session/cookies:</p>
<pre><code>apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  labels:
    app: &quot;{{ k8s.deploy.labels.app }}&quot;
    team: &quot;{{ k8s.deploy.labels.team }}&quot;
  name: &quot;{{ k8s.deploy.nginx.ingress.name }}&quot;
  namespace: &quot;{{ namespace }}&quot;
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    kubernetes.io/ingress.class: &quot;{{ k8s.ingress.class }}&quot;
    nginx.ingress.kubernetes.io/affinity: &quot;{{ k8s.ingress.affinity }}&quot;
    nginx.ingress.kubernetes.io/affinity-mode: &quot;{{ k8s.ingress.affinity.mode }}&quot;
    nginx.ingress.kubernetes.io/session-cookie-name: &quot;{{ k8s.ingress.session.cookie.name }}&quot;
    nginx.ingress.kubernetes.io/session-cookie-hash: &quot;{{ k8s.ingress.session.cookie.hash }}&quot;    
    nginx.ingress.kubernetes.io/session-cookie-expires: &quot;{{ k8s.ingress.session.cookie.expires }}&quot;
    nginx.ingress.kubernetes.io/session-cookie-max-age: &quot;{{ k8s.ingress.session.cookie.maxage }}&quot;    
spec:
  tls:
  - hosts:
      - {{ k8s.fqdn.host }}
    secretName: &quot;{{ k8s.deploy.tls.secret.name }}&quot;
  rules:
  - host: {{ k8s.fqdn.host }}
    http:
      paths:
      - backend:
          serviceName: {{ k8s.deploy.svc.name }}
          servicePort: {{ k8s.svc.http.port }}
        path: /
        pathType: &quot;{{ k8s.ingress.http.pathtype }}&quot;
</code></pre>
<p>I removed all <code>annotations</code> and it works fine now. The <code>service</code> <code>ClusterIP</code> is properly redirecting queries to new pods.</p>
