<p>My goal is to filter access by IP address of an angular app deployed on Kubernetes Engine served by nginx through a GCE ingress.</p>

<p>But on my nginx the remote_addr is not right.</p>

<p><strong>$LB_IP is the ip defined here :     kubernetes.io/ingress.global-static-ip-name: app-angular</strong></p>

<p>I'm using set_real_ip_from on nginx to set the ip from X-Forwarded-For     </p>

<pre><code>set_real_ip_from $LB_IP;
real_ip_header X-Forwarded-For;
</code></pre>

<p>The original ip comes in the X-Forwarded-For header as expected from the google doc : <a href="https://cloud.google.com/compute/docs/load-balancing/http/#components" rel="nofollow noreferrer">https://cloud.google.com/compute/docs/load-balancing/http/#components</a></p>

<p>I can see the X-Forwarded-For contains the $CLIENT_IP but the remote_addr is not correct, and by the way my filter on IP is not working. Any idea ?</p>

<p>My nginx logs :</p>

<blockquote>
  <p>10.40.40.40 - - [07/Feb/2018:11:29:48 +0000] "GET /styles.bundle.css HTTP/1.1" 200 35908 "<a href="http://MY_URL/home" rel="nofollow noreferrer">http://MY_URL/home</a>" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36" "$CLIENT_IP, $LB_IP"</p>
</blockquote>

<p>But i want : </p>

<blockquote>
  <p>$CLIENT_IP - - [07/Feb/2018:11:29:48 +0000] "GET /styles.bundle.css HTTP/1.1" 200 35908 "<a href="http://MY_URL/home" rel="nofollow noreferrer">http://MY_URL/home</a>" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36" "$CLIENT_IP, $LB_IP"</p>
</blockquote>

<p><strong>My Ingress configuration :</strong></p>

<pre><code>apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress
  annotations:
    kubernetes.io/ingress.global-static-ip-name: app-angular
spec:
  rules:
    - host: MY_URL
      http:
        paths:
        - backend:
            serviceName: app-backend
            servicePort: 80
</code></pre>

<p><strong>My backend configuration :</strong></p>

<pre><code>apiVersion: v1
kind: Service
metadata:
  name: app-backend
spec:
  type: NodePort
  selector:
    app: app-angular
  ports:
  - port: 80
    targetPort: 80
</code></pre>

<p><strong>My deployment configuration :</strong></p>

<pre><code>apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: app-angular
spec:
  replicas: 2
  revisionHistoryLimit: 2
  template:
    metadata:
      labels:
        app: app-angular
        tier: frontend
    spec:
      containers:
      - name: app-angular
        image: gcr.io/MY_PROJECT/app-angular:MY_TAG
        imagePullPolicy: Always
        ports:
          - containerPort: 80
</code></pre>

<p><strong>My nginx configuration :</strong></p>

<pre><code>server {
  listen 80;

  root /usr/share/nginx/html;

  location / {

    set_real_ip_from $LB_IP;
    real_ip_header X-Forwarded-For;

    allow 130.211.0.0/22; #Google IP
    allow $CLIENT_IP; # FILTERED IP
    allow 10.0.0.0/8; #internal network
    deny all;

    index index.html index.htm;
    try_files $uri $uri/ /index.html =404;
  }

}
</code></pre>

<p>Any idea ?</p>

<p>They are two solutions (Thks to gcbirzan for helping me on GCP slack):</p>

<p><strong>1) Update My nginx configuration with the good IPS from IP addresses ranges    :</strong></p>

<p>IP addresses ranges can be found here : <a href="https://console.cloud.google.com/networking/networks/list" rel="nofollow noreferrer">https://console.cloud.google.com/networking/networks/list</a>. 
You can just add set_real_ip_from for a region or all needed regions. 
Don't forget real_ip_recursive on;</p>

<pre><code>server {
  listen 80;

  root /usr/share/nginx/html;

  location / {

    set_real_ip_from 10.128.0.0/20;
    ...
    real_ip_header X-Forwarded-For;
    real_ip_recursive on;

    allow 130.211.0.0/22; #Google IP
    allow $CLIENT_IP; # FILTERED IP
    allow 10.0.0.0/8; #internal network
    deny all;

    index index.html index.htm;
    try_files $uri $uri/ /index.html =404;
  }

}
</code></pre>

<p><strong>2) Update My backend configuration with :</strong></p>

<p>externalTrafficPolicy: Local</p>

<pre><code>apiVersion: v1
kind: Service
metadata:
  name: app-backend
spec:
  type: NodePort
  selector:
    app: app-angular
  ports:
  - port: 80
    targetPort: 80
  externalTrafficPolicy: Local
</code></pre>

<p>Update nginx configuration :</p>

<p>now the IP shown in $remote_addr will be set with the Load balancer IP for your client requests and with google Infra IPS : 130.211.0.0/22,35.191.0.0/16 </p>

<p>Don't forget real_ip_recursive on;</p>

<pre><code>server {
  listen 80;

  root /usr/share/nginx/html;

  location / {

    set_real_ip_from $LB_IP;
    set_real_ip_from 130.211.0.0/22;
    set_real_ip_from 35.191.0.0/16;
    real_ip_header X-Forwarded-For;
    real_ip_recursive on;

    allow 130.211.0.0/22; #Google IP
    allow 35.191.0.0/16; #Google IP
    allow $CLIENT_IP; # FILTERED IP
    allow 10.0.0.0/8; #internal network
    deny all;

    index index.html index.htm;
    try_files $uri $uri/ /index.html =404;
  }

}
</code></pre>
