title: Nginx load balancer with multiple domains
body: <p>hopefully you could help me.</p>
  <p><strong>Our setup</strong></p>
  <p>Client requests a website -&gt; Route53 -&gt; Webserver</p>
  <p><strong>Where is our platform located</strong></p>
  <p>We run on GCP compute engine.</p>
  <p><strong>Our goal</strong></p>
  <p>We want to place an nginx server (which acts as a load balancer) in front of a web server so that this nginx load balancer will protect the web server from bad traffic.</p>
  <p><strong>Web server Configuration</strong></p>
  <p>Our web server has multiple domains (around 500)
  It runs nginx, apache, mysql, php-fpm, bind etc'.
  Everything on webserver runs fine.</p>
  <p><strong>The flow we expect</strong></p>
  <p>Client requests a website -&gt; Route53 -&gt; GCP HTTP/S or Network load balancer -&gt; Nginx load balancer -&gt; web server</p>
  <p><strong>What we did</strong></p>
  <p>We created an nginx server in GCP (centos7 + nginx 1.19)</p>
  <p>We created a Network Load balancer (in GCP) for port 443.</p>
  <p>We created a HTTP/S Load balancer (in GCP) for port 80.</p>
  <p>We created health checks for both GCP load balancers.</p>
  <p><strong>Configs</strong></p>
  <p>/etc/nginx/nginx.conf</p>
  <p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
  <div class="snippet-code">
  <pre class="snippet-code-html lang-html prettyprint-override"><code>user nginx;
  worker_processes auto;
  error_log /var/log/nginx/error.log;
  pid /run/nginx.pid;
  
  include /usr/share/nginx/modules/*.conf;
  
  events {
      worker_connections 124998;
      multi_accept on;
      use epoll;
  }
  
  http {
      log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                        '$status $body_bytes_sent "$http_referer" '
                        '"$http_user_agent" "$http_x_forwarded_for"';
  
      access_log  /var/log/nginx/access.log  main;
      include /etc/nginx/sites-enabled/*;
      server_names_hash_bucket_size 64;
      sendfile            on;
      tcp_nopush          on;
      tcp_nodelay         on;
      keepalive_timeout   65;
      types_hash_max_size 2048;
  
      include             /etc/nginx/mime.types;
      default_type        application/octet-stream;
  
      limit_conn_zone $binary_remote_addr zone=conn_limit_per_ip:10m;
      limit_req_zone $binary_remote_addr zone=one:10m rate=30r/m;
      open_file_cache max=400000 inactive=30s;
      open_file_cache_valid    60s;
      open_file_cache_min_uses 2;
      open_file_cache_errors   on;
  }</code></pre>
  </div>
  </div>
  </p>
  <p>a demo website config</p>
  <p>/etc/nginx/sites-available/example.com</p>
  <p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
  <div class="snippet-code">
  <pre class="snippet-code-html lang-html prettyprint-override"><code>server {
          listen 80 proxy_protocol;
          server_name example.com;
          server_name www.example.com;
  
          location / {
                  proxy_pass http://10.156.0.9:80;
                  proxy_http_version 1.1;
                  proxy_set_header Host $host;
                  proxy_set_header X-Forwarded-Proto $scheme;
                  proxy_set_header X-Forwarded-Host $server_name;
                  proxy_set_header X-Real-IP $proxy_protocol_addr;
                  proxy_set_header X-Forwarded-For $proxy_protocol_addr;
                  proxy_set_header Upgrade $http_upgrade;
                  proxy_set_header Connection "upgrade";
                  return 301 https://$host$request_uri;
          }
  }
  
  
  server {
          listen 443 ssl http2 proxy_protocol;
          server_name example.com;
          server_name www.example.com;
          ssl_stapling on;
          ssl_stapling_verify on;
          ssl_certificate_key /usr/local/psa/var/modules/letsencrypt/etc/live/example.com/privkey.pem;
          ssl_certificate /usr/local/psa/var/modules/letsencrypt/etc/live/example.com/fullchain.pem;
          ssl_prefer_server_ciphers on;
          ssl_protocols  TLSv1.2 TLSv1.3;
  
          location / {
                  proxy_pass https://10.156.0.9:443;
                  proxy_http_version 1.1;
                  proxy_set_header Host $host;
                  proxy_set_header X-Forwarded-Proto $scheme;
                  proxy_set_header X-Forwarded-Host $server_name;
                  proxy_set_header X-Real-IP $proxy_protocol_addr;
                  proxy_set_header X-Forwarded-For $proxy_protocol_addr;
          }
  }</code></pre>
  </div>
  </div>
  </p>
  <p>GCP Load balancers configs are 100% correct and work with other of our projects.</p>
  <p><strong>The problem</strong></p>
  <p>Websites are working fine when accessing <a href="https://example.com" rel="nofollow noreferrer">https://example.com</a>. But when you type <a href="http://example.com" rel="nofollow noreferrer">http://example.com</a>, it fails with error 502:</p>
  <p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
  <div class="snippet-code">
  <pre class="snippet-code-html lang-html prettyprint-override"><code>Error: Server Error
  The server encountered a temporary error and could not complete your request.
  
  Please try again in 30 seconds.</code></pre>
  </div>
  </div>
  </p>
  <p>GCP Health check for HTTP/S load balancer fails because for some reason the nginx server doesn't respond anything on port 80.</p>
  <p>So what happens?</p>
  <p>GCP Health check -&gt; checks HTTP/S load balancer -&gt; FAILS because nginx server doesn't respond anything on port 80 on its public IP (or may be private ip) -&gt; Because of that HTTP/S load balancer doesn't pass any traffic on port 80.</p>
  <p>When accessing the nginx server directly to its IP http://public_ip, it doesn't work at all.</p>
  <p>My question is. How can we make the nginx server to respond on its IP while still maintaining multiple domains server blocks?
  or how can we make that health check pass?</p>