<p>My Kubernetes 3 VM cluster (1 controller, 2 workers on Ubuntu 20.04) is having problems with DNS host lookup. It cannot correctly resolve external host names. I found this trying to run Jenkins on my cluster, and Jenkins could not get its plugins on initial setup.</p>
<p>Can anyone shed light on the following?</p>
<p>When I am using <a href="https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/" rel="nofollow noreferrer">this</a> dnsutils pod, and shelling in to do an <code>nslookup</code>, it fails.</p>
<pre><code>pod$ nslookup google.com
Server:     10.96.0.10
Address:    10.96.0.10#53

*** Can't find google.com.localdomain: No answer
</code></pre>
<p>However trying <code>dig</code> instead succeeds:</p>
<pre><code>pod$ dig google.com
; &lt;&lt;&gt;&gt; DiG 9.11.6-P1 &lt;&lt;&gt;&gt; google.com
;; global options: +cmd
;; Got answer:
;; -&gt;&gt;HEADER&lt;&lt;- opcode: QUERY, status: NOERROR, id: 10886
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 272b157caff6a2b8 (echoed)
;; QUESTION SECTION:
;google.com.            IN  A

;; ANSWER SECTION:
google.com.     5   IN  A   142.250.191.174

;; Query time: 1 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: Wed Apr 21 17:11:01 UTC 2021
;; MSG SIZE  rcvd: 77

</code></pre>
<p>If I use <code>busybox</code>, <code>nslookup</code> succeeds:</p>
<pre><code>$ kubectl run curl-busybox --image=radial/busyboxplus:curl -i --tty --rm

pod$ nslookup google.com
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      google.com
Address 1: 2607:f8b0:4009:819::200e ord38s30-in-x0e.1e100.net
Address 2: 142.250.191.174 ord38s30-in-f14.1e100.net
</code></pre>
<p>The failing case appends cluster-only search suffixes as CoreDNS emits this in the log:</p>
<pre><code>│ [INFO] 10.66.218.87:46775 - 8611 &quot;A IN google.com.default.svc.cluster.local. udp 54 false 512&quot; NXDOMAIN qr,aa,rd 147 0.00030851s                             │
│ [INFO] 10.66.218.87:55524 - 47795 &quot;A IN google.com.cluster.local. udp 42 false 512&quot; NXDOMAIN qr,aa,rd 135 0.000151907s                                       │
│ [INFO] 10.66.218.87:55367 - 64702 &quot;AAAA IN google.com.localdomain. udp 40 false 512&quot; NOERROR qr,aa 40 0.00094683s
</code></pre>
<p>The <code>/etc/resolv.conf</code> of <code>dnsutils</code> pod is:</p>
<pre><code>search default.svc.cluster.local svc.cluster.local cluster.local localdomain
nameserver 10.96.0.10
options ndots:5
</code></pre>
<p>The succeeding <code>dnsutils</code> case with <code>dig</code> emits this:</p>
<pre><code>[INFO] 10.66.218.87:59179 - 2071 &quot;A IN google.com. udp 51 false 4096&quot; NOERROR qr,rd,ra 54 0.000789572s   
</code></pre>
<p>My CoreDNS ConfigMap looks like this:</p>
<pre><code>apiVersion: v1
data:
  Corefile: |
    .:53 {
        log
        errors
        health {
           lameduck 5s
        }
        hosts /etc/coredns/customdomains.db cluster.dev {
          fallthrough
        }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
           pods insecure
           fallthrough in-addr.arpa ip6.arpa
           ttl 30
        }
        prometheus :9153
        forward . /etc/resolv.conf {
           max_concurrent 1000
        }
        cache 30
        loop
        reload
        loadbalance
    }
  customdomains.db: |
    192.168.149.130 kube-master.cluster.dev
kind: ConfigMap
metadata:
  creationTimestamp: &quot;2021-04-16T17:22:52Z&quot;
  name: coredns
  namespace: kube-system
  resourceVersion: &quot;396390&quot;
  uid: 57853d1b-3675-4686-9abe-0185f20a5bc7
</code></pre>

jws
<p>Lack of DNS forwarding on a per-pod basis appears to be <a href="https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#what-things-get-dns-names" rel="nofollow noreferrer">explained</a> by this unusual comment:</p>
<blockquote>
<p>Note: &quot;Default&quot; is not the default DNS policy. If dnsPolicy is not explicitly specified, then &quot;ClusterFirst&quot; is used.</p>
</blockquote>
<p>By adding the following to the <code>dnsutils</code> pod yaml:</p>
<pre><code>...
spec:
  ...
  dnsPolicy: Default
</code></pre>
<p>Now the pod is able to look up hostnames properly.</p>
