<p>I have been trying to use some custom, local built image on my kind cluster by following the instructions on <a href="https://kind.sigs.k8s.io/docs/user/local-registry" rel="nofollow noreferrer">https://kind.sigs.k8s.io/docs/user/local-registry</a> - i.e. applying the following <code>containerdConfigPatches</code> to my <code>cluster.cfg</code>:</p>

<pre><code>containerdConfigPatches:
- |-
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."192.168.83.82:5000"]
    endpoint = ["http://192.168.83.82:5000"]
</code></pre>

<p><code>192.168.83.82:5000</code> is the IP of the VM where the local (insecure) registry is running alongside the kind cluster and its exposed port.</p>

<p>Once the cluster is created, I can verify the settings where applied to all nodes:</p>

<pre><code>docker exec kind-worker3 cat /etc/containerd/config.toml
# [...]
# Relevant sections:
# [...]
  [plugins."io.containerd.grpc.v1.cri"]
    [plugins."io.containerd.grpc.v1.cri".registry]
      [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."192.168.83.82:5000"]
          endpoint = ["http://192.168.83.82:5000"]
</code></pre>

<p>However, pods keep failing with to <code>ErrImagePull</code> and these Event log:</p>

<pre><code>Events:
  Type     Reason     Age                From                             Message
  ----     ------     ----               ----                             -------
  Normal   Scheduled  32s                default-scheduler                Successfully assigned default/test-6bc95ff8c5-g6g86 to kind-worker3
  Normal   Pulled     31s                kubelet, kind-worker3  Container image "docker.elastic.co/beats/filebeat-oss:6.4.2" already present on machine
  Normal   Created    31s                kubelet, kind-worker3  Created container test-log-agent
  Normal   Started    31s                kubelet, kind-worker3  Started container test-log-agent
  Normal   Pulling    16s (x2 over 31s)  kubelet, kind-worker3  Pulling image "192.168.83.82:5000/test/image:2.2.1"
  Warning  Failed     16s (x2 over 31s)  kubelet, kind-worker3  Failed to pull image "192.168.83.82:5000/test/image:2.2.1": rpc error: code = Unknown desc = failed to resolve image "192.168.83.82:5000/test/image:2.2.1": no available registry endpoint: failed to do request: Head https://192.168.83.82:5000/v2/test/image/manifests/2.2.1: http: server gave HTTP response to HTTPS client
  Warning  Failed     16s (x2 over 31s)  kubelet, kind-worker3  Error: ErrImagePull
  Normal   BackOff    3s (x3 over 30s)   kubelet, kind-worker3  Back-off pulling image "192.168.83.82:5000/test/image:2.2.1"
  Warning  Failed     3s (x3 over 30s)   kubelet, kind-worker3  Error: ImagePullBackOff
</code></pre>

<p>TL;DR: "http: server gave HTTP response to HTTPS client" - which I thought would be solved with the ConfigPatch above (As it happens when you tweak <a href="https://docs.docker.com/registry/insecure/#deploy-a-plain-http-registry" rel="nofollow noreferrer">docker's <code>daemon.json</code></a>).</p>

<p>Also, as an alternative, tried loading the image from the host to the cluster nodes:</p>

<pre><code>kind load docker-image 192.168.83.82:5000/test/image:2.2.1 --name="kind-cluster"
</code></pre>

<p>And verified the image was loaded to all nodes by listing them:</p>

<pre><code>sysadmin@ubuntu:~/kind$ docker exec kind-worker3 crictl images
IMAGE                                TAG                 IMAGE ID            SIZE
192.168.83.82:5000/test/image        2.2.1               ba1601dfa9c48       822MB
docker.io/kindest/kindnetd           0.5.0               ef97cccdfdb50       83.6MB
k8s.gcr.io/coredns                   1.3.1               eb516548c180f       40.5MB
k8s.gcr.io/etcd                      3.3.10              2c4adeb21b4ff       258MB
k8s.gcr.io/kube-apiserver            v1.15.3             be321f2ded3f3       249MB
k8s.gcr.io/kube-controller-manager   v1.15.3             ac7d3fe5b34b7       200MB
k8s.gcr.io/kube-proxy                v1.15.3             d428039608992       97.3MB
k8s.gcr.io/kube-scheduler            v1.15.3             a44f53b10fee0       96.5MB
k8s.gcr.io/pause                     3.1                 da86e6ba6ca19       746kB
</code></pre>

<p>Hoping that the image now would be read from the image cache. However, the result was exactly the same. </p>

<p>Any clue on how to approach this one? Is there something I might have overlooked? </p>

<p>As Matt alluded to above, try with the insecure_skip_verify flag in your config.toml file:</p>
<pre><code>  [plugins.&quot;io.containerd.grpc.v1.cri&quot;]
    [plugins.&quot;io.containerd.grpc.v1.cri&quot;.registry]
      [plugins.&quot;io.containerd.grpc.v1.cri&quot;.registry.mirrors]
        [plugins.&quot;io.containerd.grpc.v1.cri&quot;.registry.mirrors.&quot;192.168.83.82:5000&quot;]
          endpoint = [&quot;http://192.168.83.82:5000&quot;]
      [plugins.&quot;io.containerd.grpc.v1.cri&quot;.registry.configs]
        [plugins.&quot;io.containerd.grpc.v1.cri&quot;.registry.configs.&quot;192.168.83.82:5000&quot;.tls]
          insecure_skip_verify = true
</code></pre>
