<p>Today the coredns one pod running into issue, I check the coredns pod and show log like this:</p>
<pre><code>.:53
2022-05-23T08:41:36.664Z [INFO] CoreDNS-1.3.1
2022-05-23T08:41:36.665Z [INFO] linux/amd64, go1.11.4, 6b56a9c
CoreDNS-1.3.1
linux/amd64, go1.11.4, 6b56a9c
2022-05-23T08:41:36.665Z [INFO] plugin/reload: Running configuration MD5 = 8646128cd34ade07719a0787cce6943e
2022-05-23T09:40:08.490Z [ERROR] plugin/errors: 2 oap. A: dial udp 100.100.2.136:53: i/o timeout
</code></pre>
<p>currently the coredns have 2 pod, one of the pod have this issue. The dns pod ip is <code>172.30.112.19</code>, why the dns tried to connect <code>100.100.2.136</code>? why did this happen? what should I do to make it work?</p>

<p><code>why the dns tried to connect 100.100.2.136?</code></p>
<p>When coreDNS gets request that is outside the cluster domain (eg. <a href="http://www.google.com" rel="nofollow noreferrer">www.google.com</a>), it will forward the request to upstream nameserver (likely the 100.100.2.136:53 in your case). You can check in coreDNS ConfigMap for <a href="https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#coredns-configmap-options" rel="nofollow noreferrer">forward . /etc/resolv.conf</a></p>
<p><code>why did this happen? what should I do to make it work?</code></p>
<p>If the other node that run the coreDNS pod is function correctly, there could be discrepancy in the nameserver and/or the node's network settings. It is possible also the CNI agent on the node has malfunction and messed up with IP table. You can try cordon this node and move the pod to another node.</p>
