<p>I wrote a script that starts minikube and uses <code>kubectl port-forward</code> to forward a local port to one of the services running in the kubernetes cluster. It runs <code>kubectl port-forward</code> in the background so it can run a web-server on localhost with access to a service in the cluster.</p>

<p>There's a race-condition where <code>kubectl port-forward</code> fails because the kubelet isn't ready to accept connections yet.</p>

<pre><code>error upgrading connection: error dialing backend: dial tcp 127.0.1.1:10250: getsockopt: connection refused
</code></pre>

<p>Short of putting a <code>sleep</code> in my script, how might I check whether the kubelet is ready before running the port-forward?</p>

<p>Since the error message indicates we expect the kubelet to be listening for tcp connections on port 10250, I thought I could do something like this:</p>

<pre><code>ss --listening --tcp --numeric --processes \
  '( sport = 10250 )' \
  | grep --quiet kubelet
</code></pre>

<p>But I found that even once the kubelet is ready, that <code>ss</code> command doesn't find it. I don't know if this is due to the <code>127.0.1.1</code> ip address or if there's something else I'm missing.</p>

<p>The main script, <code>run-server</code>, looks like:</p>

<pre><code>#! /usr/bin/env bash

LOCAL_PORT=5678

cleanup() {
    local pid=$(ss --listening --tcp --processes \
                   "( sport = ${LOCAL_PORT} )" \
                | grep -oP 'pid=\K\d+')

    [ -n "$pid" ] &amp;&amp; kill "$pid"
}
trap 'cleanup' EXIT

minikube start
run-port-forward $LOCAL_PORT &amp;
FOO_SERVICE_URL=localhost:${LOCAL_PORT} bin/rails server
</code></pre>

<p>and <code>run-port-forward</code> looks like</p>

<pre><code>#! /usr/bin/env bash

LOCAL_PORT=$1

pod=$(kubectl --context=minikube get pod \
      | grep -m 1 -oP "foo-service\S*")

port=$(kubectl --context=minikube get pod ${pod} -o json \
       | jq -r .spec.containers[0].ports[0].containerPort)

kubectl --context=minikube port-forward \
    ${pod} ${LOCAL_PORT}:${port}
</code></pre>

ivan
<p>Though I haven't yet found a way to check for kubelet readiness using kubectl, I came up with a workaround that performs a few retries.</p>

<p>This is the first time I've used recursion in a bash script. I got a kick out of that.</p>

<pre><code>try_port_forward() {
    retries=$1

    if (( retries &gt; 0 )); then
        sleep 1
        kubectl --context=minikube port-forward \
            ${pod} ${LOCAL_PORT}:${port} \
            2&gt;/dev/null ||
            try_port_forward $(( retries - 1 ))
    fi
}

try_port_forward 10
</code></pre>
