<p>I have cluster where I have 10 worker nodes on <code>Raspberry pi</code> and master is running on <code>Ubuntu 16.04</code>. Everything seems to be working fine but sometime a pod running on node shows below error:</p>

<p><code>Warning  FailedCreatePodSandBox  18m (x3 over 18m)   kubelet, w188  (combined from similar events): Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "6fa511fb1d292702aa99318b785f5011307601868ff2520c542515a239924c16" network for pod "deployment-6w24f": NetworkPlugin cni failed to set up pod "deployment-6w24f_aps-namespace" network: unable to allocate IP address: Post http://127.0.0.1:6784/ip/6fa511fb1d292702aa99318b785f5011307601868ff2520c542515a239924c16: dial tcp 127.0.0.1:6784: connect: connection refused</code></p>

<p>I am using weave for networking and it shows below error on kube dashboard:</p>

<p><code>Readiness probe failed: Get http://127.0.0.1:6784/status: dial tcp 127.0.0.1:6784: connect: connection refused
Back-off restarting failed container
MountVolume.SetUp failed for volume "weave-net-token-txqhk" : couldn't propagate object cache: timed out waiting for the condition</code></p>

<p>The node shows this error and after sometime it starts working fine automatically. This is happening every now and then with multiple nodes. I used below commands to init the cluster and weave:</p>

<pre><code>sudo kubeadm init --token-ttl=0 --apiserver-advertise-address=192.168.8.12

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
</code></pre>

<p>Can anyone please guide me on how to resolve this.</p>

<p>What worked for me was to add <code>iptable</code> rules with the IP range of my <code>kube-dns</code> service.</p>
<pre><code># kubectl get svc -n kube-system 
NAME       TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE
kube-dns   ClusterIP   10.96.0.10   &lt;none&gt;        53/UDP,53/TCP,9153/TCP   16m
[root@master kubernetes]# 
</code></pre>
<p>Since IP of kube-dns is <code>10.96.0.10</code> then, the valid range for iptable rule would be <code>10.96.0.1/32</code></p>
<pre><code># iptables -t nat -I KUBE-SERVICES -d 10.96.0.1/32 -p tcp -m comment --comment &quot;default/kubernetes:https cluster IP&quot; -m tcp --dport 443 -j KUBE-MARK-MASQ
</code></pre>
<p>More about this issue can be found <a href="http://www.bennythejudge.com/kubernetes/cni/weaver/2020/05/02/weave-pod-crashes-on-worker-digitalocean.html" rel="nofollow noreferrer">here</a>.</p>
