<p>After upgrading from Ubuntu 20.04 LTS to Ubuntu 22.04 LTS, I am currently facing issues with my k3s cluster.</p>
<p>For instance, the logs of the <code>local-path-provisioner</code> pod:</p>
<pre class="lang-sh prettyprint-override"><code>$ kubectl logs -n kube-system local-path-provisioner-6c79684f77-l4cqp
time=&quot;2022-04-28T03:27:00Z&quot; level=fatal msg=&quot;Error starting daemon: Cannot start Provisioner: failed to get Kubernetes server version: Get \&quot;https://10.43.0.1:443/version?timeout=32s\&quot;: dial tcp 10.43.0.1:443: i/o timeout&quot;
</code></pre>
<p>I've tried the following actions:</p>
<ul>
<li>Disabling ipv6, as described <a href="https://cwesystems.com/?p=231" rel="nofollow noreferrer">here</a></li>
<li>Disabling <code>ufw</code> firewall</li>
<li>Use legacy iptables</li>
<li>Adding rules for internal traffic to iptables, like so:</li>
</ul>
<pre class="lang-sh prettyprint-override"><code>$ sudo iptables -A INPUT -s 10.42.0.0/16 -d &lt;host_ip&gt; -j ACCEPT
</code></pre>
<p>Still, <code>coredns</code>, <code>local-path-provisioner</code> and <code>metrics-server</code> deployments won't start. When listing pods, here's the output:</p>
<pre class="lang-sh prettyprint-override"><code>$ kubectl get pods -n kube-system                                    
NAME                                      READY   STATUS             RESTARTS         AGE
cilium-64r4c                              1/1     Running            2 (18m ago)      174m
cilium-d8grw                              1/1     Running            2 (18m ago)      174m
cilium-g4gmf                              1/1     Running            2 (18m ago)      174m
cilium-h5j4h                              1/1     Running            2 (18m ago)      174m
cilium-n62nv                              1/1     Running            2 (18m ago)      174m
cilium-operator-76cff99967-6fgkv          1/1     Running            2 (18m ago)      174m
cilium-operator-76cff99967-pbr4l          1/1     Running            2 (18m ago)      174m
cilium-w4n6d                              1/1     Running            2 (18m ago)      174m
cilium-wgm7l                              1/1     Running            2 (18m ago)      174m
cilium-zqb6w                              1/1     Running            2 (18m ago)      174m
coredns-d76bd69b-bhgnl                    0/1     CrashLoopBackOff   44 (3m27s ago)   177m
hubble-relay-67f64789c7-cjzz9             0/1     CrashLoopBackOff   63 (4m15s ago)   174m
hubble-ui-794cd44b77-9vgbl                3/3     Running            6 (18m ago)      174m
local-path-provisioner-6c79684f77-l4cqp   0/1     CrashLoopBackOff   35 (3m53s ago)   177m
metrics-server-7cd5fcb6b7-v74rc           0/1     CrashLoopBackOff   42 (3m35s ago)   177m

</code></pre>
<p>Any help is appreciated! thanks</p>

Charles Guertin
<p>Since you're using Cilium, I think you might be running into this issue: <a href="https://github.com/cilium/cilium/issues/10645" rel="nofollow noreferrer">https://github.com/cilium/cilium/issues/10645</a></p>
<p>The workaround is to ensure <code>net.ipv4.conf.lxc*.rp_filter</code> is set to 0:</p>
<pre><code>echo 'net.ipv4.conf.lxc*.rp_filter = 0' | sudo tee -a /etc/sysctl.d/90-override.conf
sudo systemctl start systemd-sysctl
</code></pre>
