<p>I get this error after I start the worker node VMs(<code>Kubernetes</code>) from the <code>AWS</code> console. I am using <code>PKS</code> ( <code>Pivotal Container Service</code>)</p>

<blockquote>
  <p>network for pod "xxxxx": NetworkPlugin cni failed to set up pod "xxxxx" network: failed to set bridge addr: "cni0" already has an IP address different from 10.x.x.x/xx</p>
</blockquote>

<p>I supppose that <code>Flannel</code> assigns a subnet lease to the workers in the cluster, which expires after 24 hours - and <code>flannel.1</code> and <code>cni0 /24</code> subnet no longer match, which causes this issue. </p>

<p>I also know a workaround: </p>

<pre><code>bosh ssh -d worker -c "sudo /var/vcap/bosh/bin/monit stop flanneld" 
bosh ssh -d worker -c "sudo rm /var/vcap/store/docker/docker/network/files/local-kv.db" 
bosh ssh -d worker -c "sudo /var/vcap/bosh/bin/monit restart all"
</code></pre>

<p>However is there any permanent fix to this?</p>

<h2>TL;DR - recreate network</h2>
<pre><code>$ ip link set cni0 down
$ brctl delbr cni0  
</code></pre>
<p>Or, as @ws_ suggested in comments - remove interfaces and restart k8s services:</p>
<pre><code>ip link set cni0 down &amp;&amp; ip link set flannel.1 down 
ip link delete cni0 &amp;&amp; ip link delete flannel.1
systemctl restart containerd &amp;&amp; systemctl restart kubelet
</code></pre>
<h2>Community solutions</h2>
<p>It is a <a href="https://github.com/kubernetes/kubernetes/issues/39557" rel="noreferrer"><code>known issue</code></a></p>
<p>And there are some solutions to fix it.</p>
<p>Solution by <a href="https://github.com/kubernetes/kubernetes/issues/39557#issuecomment-271944481" rel="noreferrer">filipenv</a> is:</p>
<blockquote>
<p>on master and slaves:</p>
<pre><code>$ kubeadm reset
$ systemctl stop kubelet
$ systemctl stop docker
$ rm -rf /var/lib/cni/
$ rm -rf /var/lib/kubelet/*
$ rm -rf /etc/cni/
$ ifconfig cni0 down
$ ifconfig flannel.1 down
$ ifconfig docker0 down
</code></pre>
<p>you may need to manually <code>umount</code> filesystems from <code>/var/lib/kubelet</code> before calling rm on that dir)
After doing that I started docker and kubelet back again and restarted the kubeadm process</p>
</blockquote>
<p><a href="https://github.com/kubernetes/kubernetes/issues/39557#issuecomment-342604193" rel="noreferrer">aysark:</a> and <a href="https://github.com/feiskyer/kubernetes-handbook/blob/master/en/troubleshooting/pod.md" rel="noreferrer">kubernetes-handbook</a> in a recipe for <a href="https://github.com/feiskyer/kubernetes-handbook/blob/master/en/troubleshooting/pod.md#pod-stuck-in-waiting-or-containercreating" rel="noreferrer"><code>Pod stuck in Waiting or ContainerCreating</code></a> both recommend</p>
<pre><code>$ ip link set cni0 down
$ brctl delbr cni0  
</code></pre>
<h2>Some workarounds from Flannel's KB article</h2>
<p>And there is an article in <code>Flannel</code>'s KB: <a href="https://community.pivotal.io/s/article/PKS-Flannel-network-gets-out-of-sync-with-docker-bridge-network-cni0" rel="noreferrer">PKS Flannel network gets out of sync with docker bridge network (cni0)</a></p>
<h3>Workaround 1:</h3>
<p>WA1 is just like yours:</p>
<pre><code>    bosh ssh -d &lt;deployment_name&gt; worker -c &quot;sudo /var/vcap/bosh/bin/monit stop flanneld&quot;
    bosh ssh -d &lt;deployment_name&gt; worker -c &quot;sudo rm /var/vcap/store/docker/docker/network/files/local-kv.db&quot;
    bosh ssh -d &lt;deployment_name&gt; worker -c &quot;sudo /var/vcap/bosh/bin/monit restart all&quot;
</code></pre>
<h3>Workaround 2:</h3>
<p>If WA1 didn't help, KB recommends:</p>
<pre><code>    bosh ssh -d &lt;deployment_name&gt; worker -c &quot;sudo /var/vcap/bosh/bin/monit stop flanneld&quot;
    bosh ssh -d &lt;&gt; worker -c &quot;ifconfig | grep -A 1 flannel&quot;
    On a master node get access to etcd using the following KB 
    On a master node run `etcdctlv2 ls /coreos.com/network/subnets/`
    Remove all the worker subnet leases from etcd by running `etcdctlv2 rm /coreos.com/network/subnets/&lt;worker_subnet&gt;;` for each of the worker subnets from point 2 above.
    bosh ssh -d &lt;deployment_name&gt; worker -c &quot;sudo /var/vcap/bosh/bin/monit restart flanneld&quot;
</code></pre>
