<p>I'm trying to deploy a function with <a href="https://github.com/openfaas" rel="nofollow noreferrer">OpenFaas project</a> and a kubernetes cluster running on 2 Raspberry Pi 3B+. 
Unfortunately, the pod that should handle the function is going to <code>ImageInspectError</code> state... 
I tried to run the function with Docker directly and which is contain in a Docker image, and everything is working fine.</p>

<p>I opened an <a href="https://github.com/openfaas/faas-cli/issues/433" rel="nofollow noreferrer">issue</a> on the OpenFaas github and the maintainer told me to ask directly the Kubernetes community to get some clues.</p>

<p>My first question is :  What does ImageInspectError mean and where it comes from ?</p>

<p>And here is all the informations I have :</p>

<h2>Expected Behaviour</h2>

<p>Pod should run.</p>

<h2>Current Behaviour</h2>

<pre><code>NAMESPACE     NAME                                 READY     STATUS              RESTARTS   AGE
kube-system   etcd-masternode                      1/1       Running             1          1d
kube-system   kube-apiserver-masternode            1/1       Running             1          1d
kube-system   kube-controller-manager-masternode   1/1       Running             1          1d
kube-system   kube-dns-7f9b64f644-x42sr            3/3       Running             3          1d
kube-system   kube-proxy-wrp6f                     1/1       Running             1          1d
kube-system   kube-proxy-x6pvq                     1/1       Running             1          1d
kube-system   kube-scheduler-masternode            1/1       Running             1          1d
kube-system   weave-net-4995q                      2/2       Running             3          1d
kube-system   weave-net-5g7pd                      2/2       Running             3          1d
openfaas-fn   figlet-7f556fcd87-wrtf4              1/1       Running             0          4h
openfaas-fn   testfaceraspi-7f6fcb5897-rs4cq       0/1       ImageInspectError   0          2h
openfaas      alertmanager-66b98dd4d4-kcsq4        1/1       Running             1          1d
openfaas      faas-netesd-5b5d6d5648-mqftl         1/1       Running             1          1d
openfaas      gateway-846f8b5686-724q8             1/1       Running             2          1d
openfaas      nats-86955fb749-7vsbm                1/1       Running             1          1d
openfaas      prometheus-6ffc57bb8f-fpk6r          1/1       Running             1          1d
openfaas      queue-worker-567bcf4d47-ngsgv        1/1       Running             2          1d
</code></pre>

<p>The <code>testfaceraspi</code> doesn't run.</p>

<p>Logs from the pod : </p>

<pre><code>$ kubectl logs testfaceraspi-7f6fcb5897-rs4cq -n openfaas-fn
Error from server (BadRequest): container "testfaceraspi" in pod "testfaceraspi-7f6fcb5897-rs4cq" is waiting to start: ImageInspectError
</code></pre>

<p>Pod describe : </p>

<pre><code>$ kubectl describe pod -n openfaas-fn  testfaceraspi-7f6fcb5897-rs4cq 
Name:           testfaceraspi-7f6fcb5897-rs4cq
Namespace:      openfaas-fn
Node:           workernode/10.192.79.198
Start Time:     Thu, 12 Jul 2018 11:39:05 +0200
Labels:         faas_function=testfaceraspi
                pod-template-hash=3929761453
Annotations:    prometheus.io.scrape=false
Status:         Pending
IP:             10.40.0.16
Controlled By:  ReplicaSet/testfaceraspi-7f6fcb5897
Containers:
  testfaceraspi:
    Container ID:   
    Image:          gallouche/testfaceraspi
    Image ID:       
    Port:           8080/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       ImageInspectError
    Ready:          False
    Restart Count:  0
    Liveness:       exec [cat /tmp/.lock] delay=3s timeout=1s period=10s #success=1 #failure=3
    Readiness:      exec [cat /tmp/.lock] delay=3s timeout=1s period=10s #success=1 #failure=3
    Environment:
      fprocess:  python3 index.py
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-5qhnn (ro)
Conditions:
  Type           Status
  Initialized    True 
  Ready          False 
  PodScheduled   True 
Volumes:
  default-token-5qhnn:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-5qhnn
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  &lt;none&gt;
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason            Age                 From                 Message
  ----     ------            ----                ----                 -------
  Warning  DNSConfigForming  2m (x1019 over 3h)  kubelet, workernode  Search Line limits were exceeded, some search paths have been omitted, the applied search line is: openfaas-fn.svc.cluster.local svc.cluster.local cluster.local heig-vd.ch einet.ad.eivd.ch web.ad.eivd.ch
</code></pre>

<p>And the event logs : </p>

<pre><code>$ kubectl get events --sort-by=.metadata.creationTimestamp -n openfaas-fn
LAST SEEN   FIRST SEEN   COUNT     NAME                                              KIND         SUBOBJECT                        TYPE      REASON                  SOURCE                  MESSAGE
14m         1h           347       testfaceraspi-7f6fcb5897-rs4cq.1540db41e89d4c52   Pod                                           Warning   DNSConfigForming        kubelet, workernode     Search Line limits were exceeded, some search paths have been omitted, the applied search line is: openfaas-fn.svc.cluster.local svc.cluster.local cluster.local heig-vd.ch einet.ad.eivd.ch web.ad.eivd.ch
4m          1h           75        figlet-7f556fcd87-wrtf4.1540db421002b49e          Pod                                           Warning   DNSConfigForming        kubelet, workernode     Search Line limits were exceeded, some search paths have been omitted, the applied search line is: openfaas-fn.svc.cluster.local svc.cluster.local cluster.local heig-vd.ch einet.ad.eivd.ch web.ad.eivd.ch
10m         10m          1         testfaceraspi-7f6fcb5897-d6z78.1540df9ed8b91865   Pod                                           Normal    Scheduled               default-scheduler       Successfully assigned testfaceraspi-7f6fcb5897-d6z78 to workernode
10m         10m          1         testfaceraspi-7f6fcb5897.1540df9ed6eee11f         ReplicaSet                                    Normal    SuccessfulCreate        replicaset-controller   Created pod: testfaceraspi-7f6fcb5897-d6z78
10m         10m          1         testfaceraspi-7f6fcb5897-d6z78.1540df9eef3ef504   Pod                                           Normal    SuccessfulMountVolume   kubelet, workernode     MountVolume.SetUp succeeded for volume "default-token-5qhnn" 
4m          10m          27        testfaceraspi-7f6fcb5897-d6z78.1540df9eef5445c0   Pod                                           Warning   DNSConfigForming        kubelet, workernode     Search Line limits were exceeded, some search paths have been omitted, the applied search line is: openfaas-fn.svc.cluster.local svc.cluster.local cluster.local heig-vd.ch einet.ad.eivd.ch web.ad.eivd.ch
8m          9m           8         testfaceraspi-7f6fcb5897-d6z78.1540df9f670d0dad   Pod          spec.containers{testfaceraspi}   Warning   InspectFailed           kubelet, workernode     Failed to inspect image "gallouche/testfaceraspi": rpc error: code = Unknown desc = Error response from daemon: readlink /var/lib/docker/overlay2/l: invalid argument
9m          9m           7         testfaceraspi-7f6fcb5897-d6z78.1540df9f670fcf3e   Pod          spec.containers{testfaceraspi}   Warning   Failed                  kubelet, workernode     Error: ImageInspectError
</code></pre>

<h2>Steps to Reproduce (for bugs)</h2>

<ol>
<li>Deploy OpenFaas on a 2 node k8s cluster</li>
<li>Create function with <code>faas new testfaceraspi --lang python3-armhf</code></li>
<li><p>Add the following code in the <code>handler.py</code> : </p>

<pre><code>import json
def handle(req):
    jsonl = json.loads(req)

    return ("Found " + str(jsonl["nbFaces"]) + " faces in OpenFaas Function on raspi !")
</code></pre></li>
<li><p>Change gateway and image in the <code>.yml</code>
    provider:
      name: faas
      gateway: <a href="http://127.0.0.1:31112" rel="nofollow noreferrer">http://127.0.0.1:31112</a></p>

<pre><code>functions:
  testfaceraspi:
    lang: python3-armhf
    handler: ./testfaceraspi
    image: gallouche/testfaceraspi
</code></pre></li>
<li><p>Run <code>faas build -f testfacepi.yml</code></p></li>
<li>Login in DockerHub with <code>docker login</code></li>
<li>Run <code>faas push -f testfacepi.yml</code></li>
<li>Run <code>faas deploy -f testfacepi.yml</code></li>
</ol>

<h2>Your Environment</h2>

<ul>
<li><p>FaaS-CLI version ( Full output from: <code>faas-cli version</code> ):</p>

<pre><code>Commit: 3995a8197f1df1ecdf524844477cffa04e4690ea
Version: 0.6.11
</code></pre></li>
<li><p>Docker version ( Full output from: <code>docker version</code> ):</p>

<pre><code>Client:
 Version:       18.04.0-ce
 API version:   1.37
 Go version:    go1.9.4
 Git commit:    3d479c0
 Built: Tue Apr 10 18:25:24 2018
 OS/Arch:       linux/arm
 Experimental:  false
 Orchestrator:  swarm

 Server:
 Engine:
  Version:      18.04.0-ce
  API version:  1.37 (minimum version 1.12)
  Go version:   go1.9.4
  Git commit:   3d479c0
  Built:        Tue Apr 10 18:21:25 2018
  OS/Arch:      linux/arm
  Experimental: false
</code></pre></li>
<li><p>Operating System and version (e.g. Linux, Windows, MacOS):</p>

<pre><code>Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 9.4 (stretch)
Release:        9.4
Codename:       stretch
</code></pre></li>
</ul>

<p>Thanks by advance and tell me if you need any more informations.</p>

<p>Gallouche</p>

Gallouche
<p>I've seen this error because the docker version wasn't supported by Kubernetes. As of Kubernetes version 1.11, the supported versions are 1.11.2 to 1.13.1 and 17.03.x.</p>

<p>I couldn't test the solution with OpenFaaS.</p>
