<p>I followed some tutorials on how to set up an HTTP server, and test it in a local Kubernetes cluster (using <code>minikube</code>).</p>

<p>I also implemented graceful shutdown from some examples I found, and expected that there would be no downtime from a Kubernetes rolling restart.</p>

<p>To verify that, I started performing load tests (using <a href="https://httpd.apache.org/docs/2.4/programs/ab.html" rel="nofollow noreferrer">Apache Benchmark</a>, by running <code>ab -n 100000 -c 20 &lt;addr&gt;</code>) and running <code>kubectl rollout restart</code> during the benchmarking, but <code>ab</code> stops running as soon as the rolling restart is performed.</p>

<p>Here is my current project setup:</p>

<p><strong>Dockerfile</strong></p>

<pre><code>FROM golang:1.13.4-alpine3.10

RUN mkdir /app
ADD . /app
WORKDIR /app

RUN go build -o main src/main.go
CMD ["/app/main"]
</code></pre>

<p><strong>src/main.go</strong></p>

<pre class="lang-golang prettyprint-override"><code>package main

import (
    "context"
    "fmt"
    "log"
    "net/http"
    "os"
    "os/signal"
    "syscall"

    "github.com/gorilla/mux"
)

func main() {
    srv := &amp;http.Server{
        Addr:    ":8080",
        Handler: NewHTTPServer(),
    }

    idleConnsClosed := make(chan struct{})
    go func() {
        sigint := make(chan os.Signal, 1)
        signal.Notify(sigint, os.Interrupt, syscall.SIGTERM, syscall.SIGINT)
        &lt;-sigint

        // We received an interrupt signal, shut down.
        if err := srv.Shutdown(context.Background()); err != nil {
            // Error from closing listeners, or context timeout:
            log.Printf("HTTP server Shutdown: %v", err)
        }

        close(idleConnsClosed)
    }()

    log.Printf("Starting HTTP server")
    running = true
    if err := srv.ListenAndServe(); err != http.ErrServerClosed {
        // Error starting or closing listener:
        log.Fatalf("HTTP server ListenAndServe: %v", err)
    }

    &lt;-idleConnsClosed
}

func NewHTTPServer() http.Handler {
    r := mux.NewRouter()

    // Ping
    r.HandleFunc("/", handler)

    return r
}

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hello World!")
}
</code></pre>

<p><strong>kubernetes/deployment.yaml</strong></p>

<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: myapp
  name: myapp
spec:
  replicas: 10
  selector:
    matchLabels:
      app: myapp
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 5
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: myapp
        image: dickster/graceful-shutdown-test:latest
        imagePullPolicy: Never
        ports:
        - containerPort: 8080

</code></pre>

<p><strong>kubernetes/service.yaml</strong></p>

<pre><code>apiVersion: v1
kind: Service
metadata:
  labels:
    app: myapp
  name: myapp
spec:
  ports:
  - port: 8080
    protocol: TCP
  selector:
    app: myapp
  sessionAffinity: None
  type: NodePort
</code></pre>

<p>Is there something missing in this setup? According to the <code>rollingUpdate</code> strategy, there should be at least five running pods that should serve the incoming requests, but <code>ab</code> exits with an <code>apr_socket_recv: Connection reset by peer (54)</code> error. I also tried adding readiness/liveness probes, but no luck. I suspect they're not needed here, either.</p>

<p>For this to work without downtime, you need to have the pods stop receiving new connections while the pod is allowed to gracefully finish handling current connections. This means the pod needs to be running, but not ready so that new requests are not sent to it.</p>

<p>Your service will match all pods using the label selector you configured (I assume <code>app: myapp</code>) and will use any pod in the ready state as a possible backend. The pod is marked as ready as long as it is passing the readinessProbe. Since you have no probe configured, the pod status will default to ready as long as it is running.</p>

<p>Just having a <a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-readiness-probes" rel="nofollow noreferrer">readinessProbe</a> configured will help immensely, but will not provide 100% uptime, that will require some tweaks in your code to cause the readinessProbe to fail (so new requests are not sent) while the container gracefully finishes with current connections.</p>

<p>EDIT: As per @Thomas Jungblut mentioned, a big part of eliminating errors with your webserver is how the application handles SIGTERM. While the pod is in terminating state, it will no longer receive requests through the service. During this phase, your webserver needs to be configured to gracefully complete and terminate connections rather than stop abruptly and terminate requests. </p>

<p>Note that this is configured in the application itself and is not a k8s setting. As long as the webserver gracefully drains the connections and your pod spec includes a gracefulTerminationPeriod long enough to allow the webserver to drain, you should see basically no errors. Although this still won't guarantee 100% uptime, especially when bombarding the service using ab.</p>
