<p>Follow <a href="https://blog.hasura.io/sharing-a-local-registry-for-minikube-37c7240d0615" rel="nofollow noreferrer">this</a> guide, I'm trying to start <code>minikube</code> and forward port at the boot time.</p>

<p>My script:</p>

<pre class="lang-sh prettyprint-override"><code>#!/bin/bash

set -eux

export PATH=/usr/local/bin:$PATH

minikube status || minikube start
minikube ssh 'grep docker.for.mac.localhost /etc/hosts || echo -e "127.0.0.1\tdocker.for.mac.localhost" | sudo tee -a /etc/hosts'
minikube ssh 'test -f wait-for-it.sh || curl -O https://raw.githubusercontent.com/vishnubob/wait-for-it/master/wait-for-it.sh'
minikube ssh 'chmod +x wait-for-it.sh &amp;&amp; ./wait-for-it.sh 127.0.1.1:10250'
POD=$(kubectl get po --namespace kube-system | awk '/kube-registry-v0/ { print $1 }')
kubectl port-forward --namespace kube-system $POD 5000:5000
</code></pre>

<p>Everything works fine except that <code>kubectl port-forward</code> said that pod does not exist at the first time running:</p>

<pre class="lang-sh prettyprint-override"><code>++ kubectl get po --namespace kube-system
++ awk '/kube-registry-v0/ { print $1 }'
+ POD=kube-registry-v0-qr2ml
+ kubectl port-forward --namespace kube-system kube-registry-v0-qr2ml 5000:5000
error: error upgrading connection: unable to upgrade connection: pod does not exist
</code></pre>

<p>If I re-run:</p>

<pre class="lang-sh prettyprint-override"><code>+ minikube status
minikube: Running
cluster: Running
kubectl: Correctly Configured: pointing to minikube-vm at 192.168.99.100
+ minikube ssh 'grep docker.for.mac.localhost /etc/hosts || echo -e "127.0.0.1\tdocker.for.mac.localhost" | sudo tee -a /etc/hosts'
127.0.0.1   docker.for.mac.localhost
+ minikube ssh 'test -f wait-for-it.sh || curl -O https://raw.githubusercontent.com/vishnubob/wait-for-it/master/wait-for-it.sh'
+ minikube ssh 'chmod +x wait-for-it.sh &amp;&amp; ./wait-for-it.sh 127.0.1.1:10250'
wait-for-it.sh: waiting 15 seconds for 127.0.1.1:10250
wait-for-it.sh: 127.0.1.1:10250 is available after 0 seconds
++ kubectl get po --namespace kube-system
++ awk '/kube-registry-v0/ { print $1 }'
+ POD=kube-registry-v0-qr2ml
+ kubectl port-forward --namespace kube-system kube-registry-v0-qr2ml 5000:5000
Forwarding from 127.0.0.1:5000 -&gt; 5000
Forwarding from [::1]:5000 -&gt; 5000
</code></pre>

<p>I added a debug line before forwarding:</p>

<pre class="lang-sh prettyprint-override"><code>kubectl describe pod --namespace kube-system $POD
</code></pre>

<p>and saw this:</p>

<pre class="lang-sh prettyprint-override"><code>+ POD=kube-registry-v0-qr2ml
+ kubectl describe pod --namespace kube-system kube-registry-v0-qr2ml
Name:       kube-registry-v0-qr2ml
Namespace:  kube-system
Node:       minikube/192.168.99.100
Start Time: Thu, 28 Dec 2017 10:00:00 +0700
Labels:     k8s-app=kube-registry
        version=v0
Annotations:    kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"ReplicationController","namespace":"kube-system","name":"kube-registry-v0","uid":"317ecc42-eb7b-11e7-a8ce-...
Status:     Running
IP:     172.17.0.6
Controllers:    ReplicationController/kube-registry-v0
Containers:
  registry:
    Container ID:   docker://6e8f3f33399605758354f3f546996067d834459781235d51eef3ffa9c6589947
    Image:      registry:2.5.1
    Image ID:       docker-pullable://registry@sha256:946480a23b33480b8e7cdb89b82c1bd6accae91a8e66d017e21e8b56551f6209
    Port:       5000/TCP
    State:      Running
      Started:      Thu, 28 Dec 2017 13:22:44 +0700
</code></pre>

<p>Why <code>kubectl</code> said that it does not exist?</p>

<hr>

<p><strong>Fri Dec 29 04:58:06 +07 2017</strong></p>

<p>Looking carefully at the events, I found something:</p>

<pre><code>Events:
  FirstSeen     LastSeen        Count   From                    SubObjectPath                   Type            Reason                  Message
  ---------     --------        -----   ----                    -------------                   --------        ------                  -------
  20m           20m             1       kubelet, minikube                                       Normal          SuccessfulMountVolume   MountVolume.SetUp succ
eeded for volume "image-store"
  20m           20m             1       kubelet, minikube                                       Normal          SuccessfulMountVolume   MountVolume.SetUp succ
eeded for volume "default-token-fs7kr"
  20m           20m             1       kubelet, minikube                                       Normal          SandboxChanged          Pod sandbox changed, it will be killed and re-created.
  20m           20m             1       kubelet, minikube       spec.containers{registry}       Normal          Pulled                  Container image "registry:2.5.1" already present on machine
  20m           20m             1       kubelet, minikube       spec.containers{registry}       Normal          Created                 Created container
  20m           20m             1       kubelet, minikube       spec.containers{registry}       Normal          Started                 Started container
</code></pre>

<blockquote>
  <p>Pod sandbox changed, it will be killed and re-created.</p>
</blockquote>

<p>Before:</p>

<pre><code>Containers:
  registry:
    Container ID:       docker://47c510dce00c6c2c29c9fe69665e1241c457d0666174a7723062c534e7229c58
    Image:              registry:2.5.1
    Image ID:           docker-pullable://registry@sha256:946480a23b33480b8e7cdb89b82c1bd6accae91a8e66d017e21e8b56551f6209
    Port:               5000/TCP
    State:              Running
      Started:          Thu, 28 Dec 2017 13:47:02 +0700
    Last State:         Terminated
      Reason:           Error
      Exit Code:        2
      Started:          Thu, 28 Dec 2017 13:22:44 +0700
      Finished:         Thu, 28 Dec 2017 13:45:18 +0700
    Ready:              True
    Restart Count:      14
</code></pre>

<p>After:</p>

<pre><code>Containers:
  registry:
    Container ID:       docker://3a7da784d3d596796111348757725f5af22b47c5edd0fc29a4ffbb84f3f08956
    Image:              registry:2.5.1
    Image ID:           docker-pullable://registry@sha256:946480a23b33480b8e7cdb89b82c1bd6accae91a8e66d017e21e8b56551f6209
    Port:               5000/TCP
    State:              Running
      Started:          Thu, 28 Dec 2017 19:03:04 +0700
    Last State:         Terminated
      Reason:           Error
      Exit Code:        2
      Started:          Thu, 28 Dec 2017 13:47:02 +0700
      Finished:         Thu, 28 Dec 2017 19:00:48 +0700
    Ready:              True
    Restart Count:      15
</code></pre>

<p>minikube logs:</p>

<blockquote>
  <p>Dec 28 22:15:41 minikube localkube[3250]: W1228 22:15:41.102038<br>
  3250 docker_sandbox.go:343] failed to read pod IP from plugin/docker:
  Couldn't find network status for kube-system/kube-registry-v0-qr2ml
  through plugin: invalid network status for</p>
</blockquote>

<blockquote>
  <p>POD=$(kubectl get po --namespace kube-system | awk '/kube-registry-v0/ { print $1 }')</p>
</blockquote>

<p>Be aware that using a <a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/" rel="noreferrer">selector</a> is almost certainly better than using text utilities, especially with "unstructured" output from <code>kubectl</code>. I don't know of any promises they make about the format of the default output, which is why <code>--output=json</code> and friends exist. However, in your case when you just want the name, there is a special <code>--output=name</code> which does what it says, with the mild caveat that the Resource prefix will be in front of the name (<code>pods/kube-registry-v0-qr2ml</code> in your case)</p>

<p>Separately, I see that you have "wait-for-it," but just because a port is accepting connections doesn't mean the Pod is <strong>Ready</strong>. You'll actually want to use <code>--output=json</code> (or more <code>awk</code> scripts, I guess) to ensure the Pod is both Running and Ready, with the latter status reached when kubernetes and the Pod agree that everything is cool.</p>

<p>I <em>suspect</em>, but would have to experiment to know for sure, that the error message is just misleading; it isn't truly that kubernetes doesn't know anything about your Pod, but merely that it couldn't port-forward to it in the state it's in.</p>

<p>You may also experience better success by creating a <code>Service</code> of <code>type: NodePort</code> and then talk to the Node's IP on the allocated port; that side-steps this kubectl-shell mess entirely, but does <strong>not</strong> side-step the Ready part -- only Pods in the Ready state will receive traffic from a Service</p>

<hr>

<p>As a minor, pedantic note, <code>--namespace</code> is an argument to <code>kubectl</code>, and not to <code>port-forward</code>, so the most correct invocation is <code>kubectl --namespace=kube-system port-forward kube-registry-v0-qr2ml 5000:5000</code> to ensure the argument isn't mis-parsed</p>
