<p>I have a local 3-nodes-cluster of Kubernetes, with weave-Net as the Overlay-Network-Plugin installed. My goal is to try out network policies of kubernetes and to export log messages of this process towards an elk-stack.</p>

<p>Unfortunately, I cannot proceed, because I cannot solve my issues with kube-dns. The name resolution seems to work, but the network connection from a pod to a service is problematic.</p>

<p>Here some facts about my setup (See below for versions/general config details):</p>

<ul>
<li>I am logged in to a pod "<code>busybox</code>"</li>
<li>I have a service called "<code>nginx</code>", connected with an nginx-pod that is up and running</li>
<li>From the busybox, I cannot ping the dns:<code>25 packets transmitted, 0 packets received, 100% packet loss</code></li>
<li><p>If I try "<code>nslookup nginx</code>" I get:</p>

<pre><code>Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      nginx
Address 1: 10.104.126.131 nginx.default.svc.cluster.local
</code></pre></li>
<li><p>Also I Changed changed the config-File on the busybox-Pod manually, to make the name resolution without FQDN:</p>

<pre><code>/ # cat /etc/resolv.conf
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local nginx XXX.XXX.de
options ndots:5
</code></pre>

<p>This doesn't seem like a good workaround for me, but at least it's working, and the nslookup is giving me the correct IP of the nginx-Service:</p>

<pre><code>user@controller:~$ kubectl get svc
NAME                     TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE

nginx                    ClusterIP   10.104.126.131   &lt;none&gt;        80/TCP         3d
</code></pre></li>
<li><p>Now, back to my networking issue: There doesn't seem to be the correct network interface on the pod that the connection to the service can be established:</p>

<pre><code>/ # ifconfig
eth0      Link encap:Ethernet  HWaddr 02:42:AC:11:00:02
          inet addr:172.17.0.2  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::42:acff:fe11:2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:272 errors:0 dropped:0 overruns:0 frame:0
          TX packets:350 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:23830 (23.2 KiB)  TX bytes:32140 (31.3 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
</code></pre></li>
</ul>

<p>The busybox-pod has this IP: <code>172.17.0.2</code> while the DNS is in the subnet that starts with 10. IP of dns: <code>10.96.0.10</code>.</p>

<ul>
<li>Weavenet sometimes crashes on one worker-Node, but in the general case it is shown as "running" and I don't think that this can be the reason.</li>
</ul>

<p>--> Can anybody see the underlaying configuration mistake in my networking? I'd be glad for hints! :)</p>

<p><em>General information:</em></p>

<p>Kubernetes/kubectl:  <code>v1.9.2</code></p>

<p>I used kubeadm to install.</p>

<p>uname -a: <code>Linux controller 4.4.0-87-generic #110-Ubuntu SMP Tue Jul 18 12:55:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux</code></p>

<p>docker version:</p>

<pre><code>    Client:
     Version:      1.13.1
     API version:  1.26
     Go version:   go1.6.2
     Git commit:   092cba3
     Built:        Thu Nov  2 20:40:23 2017
     OS/Arch:      linux/amd64

    Server:
     Version:      1.13.1
     API version:  1.26 (minimum version 1.12)
     Go version:   go1.6.2
     Git commit:   092cba3
     Built:        Thu Nov  2 20:40:23 2017
     OS/Arch:      linux/amd64
     Experimental: false
</code></pre>

<p>Weavenet: <code>2.2.0</code></p>

<p><strong>None of the Service IPs (aka. Cluster IPs, aka. Portal IPs) respond to ping</strong>, so that's not a good test! A good test is to try the Service IP with an appropriate client, like nslookup for dns, curl for http, etc (and make sure you do that on the correct port too).</p>

<p>As you've seen from <code>nslookup</code>, the kube-dns Service is functioning properly.</p>

<p>There is a very good document about how these <a href="https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies" rel="nofollow noreferrer">virtual IPs</a> work in Kubernetes. Long story short: you will not find a network interface for this network, it is created via redirections in the kernel (configured by iptables).</p>
