<p>I spinned up k8 cluster using Kesley KTHW (<a href="https://github.com/kelseyhightower/kubernetes-the-hard-way/" rel="nofollow noreferrer">https://github.com/kelseyhightower/kubernetes-the-hard-way/</a>) in GCP.</p>
<p>Trying to do some exercise on this link =&gt; <a href="https://github.com/dgkanatsios/CKAD-exercises/blob/master/b.multi_container_pods.md" rel="nofollow noreferrer">https://github.com/dgkanatsios/CKAD-exercises/blob/master/b.multi_container_pods.md</a> and my external dns resolution fails from the pod.</p>
<p><strong>Version:</strong></p>
<pre><code>sshanmugagani@MSI:~/cka/skk8/practise-1$ kubectl version Client Version: version.Info{Major:&quot;1&quot;, Minor:&quot;18&quot;, GitVersion:&quot;v1.18.6&quot;, GitCommit:&quot;dff82dc0de47299ab66c83c626e08b245ab19037&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2020-07-15T16:58:53Z&quot;, GoVersion:&quot;go1.13.9&quot;, Compiler:&quot;gc&quot;, Platform:&quot;linux/amd64&quot;} Server Version: version.Info{Major:&quot;1&quot;, Minor:&quot;18&quot;, GitVersion:&quot;v1.18.6&quot;, GitCommit:&quot;dff82dc0de47299ab66c83c626e08b245ab19037&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2020-07-15T16:51:04Z&quot;, GoVersion:&quot;go1.13.9&quot;, Compiler:&quot;gc&quot;, Platform:&quot;linux/amd64&quot;}
</code></pre>
<p><strong>Pod fails to resolve google.com:</strong></p>
<pre><code>sshanmugagani@MSI:~/cka/skk8/practise-1$ kubectl exec -ti dnsutils -- nslookup google
Server: 10.32.0.10
Address: 10.32.0.10#53

** server can't find google.us-west1-c.c.test.internal: SERVFAIL

command terminated with exit code 1
</code></pre>
<p><strong>Pod's /etc/resolv.conf:</strong></p>
<pre><code>sshanmugagani@MSI:~/cka/skk8/practise-1$ kubectl apply -f https://k8s.io/examples/admin/dns/dnsutils.yaml pod/dnsutils created 
</code></pre>
<p>sshanmugagani@MSI:~/cka/skk8/practise-1$ k exec -it dnsutils -- cat /etc/resolv.conf
search default.svc.cluster.local svc.cluster.local cluster.local us-west1-c.c.test.internal c.test.internal google.internal
nameserver 10.32.0.10
options ndots:5</p>
<p><strong>Getting worker node where pod runs:</strong></p>
<pre><code>sshanmugagani@MSI:~/cka/skk8/practise-1$ kgp -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES dnsutils 1/1 Running 0 60s 10.200.0.65 worker-0 &lt;none&gt; &lt;none&gt; multi 0/2 Completed 0 12h 10.200.0.53 worker-0 &lt;none&gt; &lt;none&gt;
</code></pre>
<p><strong>Worker node resolves:</strong></p>
<pre><code>sshanmugagani@worker-0:~$ nslookup google.com
Server: 127.0.0.53
Address: 127.0.0.53#53

Non-authoritative answer:
Name: google.com
Address: 74.125.20.101
Name: google.com
Address: 74.125.20.100
Name: google.com
Address: 74.125.20.139
Name: google.com
Address: 74.125.20.138
Name: google.com
Address: 74.125.20.113
Name: google.com
Address: 74.125.20.102
Name: google.com
Address: 2607:f8b0:400e:c09::65
Name: google.com
Address: 2607:f8b0:400e:c09::8a
Name: google.com
Address: 2607:f8b0:400e:c09::8b
Name: google.com
Address: 2607:f8b0:400e:c09::71
</code></pre>
<p><strong>Coredns:</strong></p>
<pre><code>sshanmugagani@MSI:~/cka/skk8/practise-1$ kgp $ks NAME READY STATUS RESTARTS AGE coredns-5677dc4cdb-cfl2j 1/1 Running 1 11h coredns-5677dc4cdb-xqm44 1/1 Running 1 11h
</code></pre>
<p><strong>Coredns logs:</strong></p>
<pre><code>sshanmugagani@MSI:~/cka/skk8/practise-1$ kubectl logs coredns-5677dc4cdb-cfl2j $ks .:53 [INFO] plugin/reload: Running configuration MD5 = fbb756dad13bce75afc40db627b38529 CoreDNS-1.7.0 linux/amd64, go1.14.4, f59c03d [ERROR] plugin/errors: 2 2953017454530458158.338294255644342916. HINFO: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 google.com.us-west1-c.c.test.internal. A: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 google.com. A: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 google.com. A: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 kube-dns. A: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 neverssl.com.us-west1-c.c.test.internal. AAAA: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 neverssl.com.c.test.internal. AAAA: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 neverssl.com. AAAA: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 neverssl.com.c.test.internal. AAAA: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 neverssl.com.goo. AAAA: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 neverssl.com.us-west1-c.c.test.internal. AAAA: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 neverssl.com.c.test.internal. AAAA: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 neverssl.com.goo. AAAA: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 neverssl.com. A: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 neverssl.com.goo. A: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 neverssl.com. A: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 neverssl.com.us-west1-c.c.test.internal. A: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 neverssl.com.c.test.internal. A: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 neverssl.com.goo. A: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 neverssl.com.us-west1-c.c.test.internal. A: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 google.com. A: plugin/loop: no next plugin found [ERROR] plugin/errors: 2 google. A: plugin/loop: no next plugin found
</code></pre>

<p>Got this issue fixed by
==&gt; Loaded kernel module
modprobe br_netfilter
===&gt; Add flag for kube-proxy start up file
--masquerade-all</p>
