<p>In trying to achieve a highly available web server set up, I ran a load test against an extremely simple NGINX docker container serving a single static file.</p>

<p>At first, I tried with a single node (n1-standard-4) &amp; single pod cluster, to benchmark how much one "unit" could do. This single node/pod setup could cope with around 20k concurrent requests before starting to timeout for some requests/drop in throughput.</p>

<p>I then added another node of the same machine type and scaled the pods to two replicas. Once I confirmed both pods/nodes were up, I ran the test again with 20k. Performance was good so I bumped up to 40k - expecting that I would see similar results to the 20k on the 1 node/pod setup.</p>

<p>However, the performance was very bad, never with requests per second jumping violently between 15k and 30k.</p>

<p>I tried the same test again with 4 nodes/pods and saw similar, if not slightly worse results.</p>

<p>My question(s) are:</p>

<ul>
<li><p>Am I wrong to think that my concurrent requests should scale linearly this way with GKE/containers/kubernetes?</p></li>
<li><p>What am I missing in order to achieve the desired results of being able to cope with N concurrent users with M nodes/pods in the cluster?</p></li>
</ul>

<p>EDIT: I also do not believe it is an issue with the load testing tool - as I am using an external paid service that claims to be able to simulate up to 100k concurrent requests.</p>

<p>EDIT 2: Here's some more information about the setup:</p>

<p><strong>Dockerfile:</strong></p>

<pre><code>FROM nginx

ADD nginx.conf /etc/nginx/nginx.conf
ADD index.html /usr/share/nginx/html/index.html
ADD data.json /usr/share/nginx/html/data.json
</code></pre>

<p><strong>nginx.conf:</strong></p>

<pre><code>user  nginx;
worker_processes  4;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  4096;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';

    access_log off;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;


    server {
        listen 80 default_server;
        listen [::]:80 default_server;

        root /usr/share/nginx/html;

        # Serve the index.html page
        location / {
            try_files /index.html =404;
        }
    }

    include /etc/nginx/conf.d/*.conf;
}
</code></pre>

<p><strong>index.html:</strong></p>

<pre><code>&lt;head&gt;&lt;title&gt;Load Test Page&lt;/title&gt;&lt;/head&gt;&lt;body&gt;&lt;h3&gt;Load Test!&lt;/h3&gt;&lt;/body&gt;
</code></pre>

<p>I used the "LoadBalancer" service, which I believe setups a Google Cloud Balancer (I confirmed this and saw the nodes being added to it). Here are the files below I used with kubernetes to manage the cluster.</p>

<p><strong>rc.yml:</strong></p>

<pre><code>apiVersion: v1
kind: ReplicationController
metadata:
  name: nginx-rc
  labels:
    name: nginx-rc
spec:
  replicas: 2
  selector:
    name: nginx-server
  template:
    metadata:
      labels:
        name: nginx-server
      name: nginx-server
    spec:
      containers:
      - name: nginx-server
        image: [[ my image ]]
        ports:
        - containerPort: 80
          hostPort: 80
</code></pre>

<p><strong>services.yml:</strong></p>

<pre><code>apiVersion: v1
kind: Service
metadata:
  labels:
    name: nginx-lb
  name: nginx-lb
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    name: nginx-server
  type: LoadBalancer
</code></pre>

<p>If you want to run a preconfigured load test, you can check out the instructions for the <a href="https://github.com/kubernetes/contrib/tree/master/scale-demo" rel="nofollow">kubernetes scale-demo</a> that shows how to serve 1 million QPS of static files using nginx, which is very similar to your test setup. </p>

<p>Also keep in mind that a single VM (regardless of the number of pods running on it) will have a limit to <a href="https://cloud.google.com/compute/docs/networks-and-firewalls#egress_throughput_caps" rel="nofollow">how much bandwidth</a> it can use and how many packets-per-second it can process. To serve more load, you should create a larger VM type or add additional VMs.</p>
