<p>I've install a Kubernetes 1.2.4 on 3 minons/master (1 master/minion, 2 minions) and installed the SkyDNS addons. After fixing SSL cert problems, I know have SkyDNS working. But kubeletes still says that I didn't set cluster-dns and cluster-domain.</p>

<p>(see edits at the bottom)</p>

<p>But you can see <code>--cluster-dns=192.168.0.10 --cluster-domain=cluster.local</code>:</p>

<pre><code>ps ax | grep kubelet
18717 ?        Ssl    0:04 /opt/kubernetes/bin/kubelet --logtostderr=true --v=0 --address=0.0.0.0 --port=10250 --hostname-override=k8s-minion-1 --api-servers=http://k8s-master:8080 --allow-privileged=false  --cluster-dns=192.168.0.10 --cluster-domain=cluster.local 
</code></pre>

<p>Launching this pod:</p>

<pre><code>apiVersion: v1
kind: Pod
metadata:
  name: busybox
  namespace: default
spec:
  containers:
  - image: busybox
    command:
      - sleep
      - "3600"
    imagePullPolicy: IfNotPresent
    name: busybox
  restartPolicy: Always
</code></pre>

<p>I see:</p>

<pre><code>kubectl describe pod busybox
7m      7m      2   {kubelet k8s-master.XXX}                    Warning     MissingClusterDNS   kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy.
</code></pre>

<p>I restarted kubelete services before to launch this pod, and I have no other pod running.</p>

<p>If I launch docker container using "--dns" option:</p>

<pre><code>docker run --rm -it --dns 192.168.0.10 busybox nslookup cluster.local
Server:    192.168.0.10
Address 1: 192.168.0.10

Name:      cluster.local
Address 1: 192.168.0.10
Address 2: 172.16.50.2
Address 3: 192.168.0.1
Address 4: 172.16.96.3


docker run --rm -it --dns 192.168.0.10 busybox cat /etc/resolv.conf
search XXX YYYY 
nameserver 192.168.0.10
</code></pre>

<p>That's absolutly normal (I've hidden my client dns)</p>

<p>But the pod says something else:</p>

<pre><code>kubectl exec busybox -- nslookup cluster.local
Server:    XXX.YYY.XXX.YYY
Address 1: XXX.YYYY.XXXX.YYY XXX.domain.fr

nslookup: can't resolve 'cluster.local'
error: error executing remote command: Error executing command in container: Error executing in Docker Container: 1
</code></pre>

<p>I tried to set "--dns" option to the docker daemon, but the error is the same.</p>

<p>See that logs:</p>

<pre><code>kubectl get  pods --namespace=kube-system
NAME                 READY     STATUS    RESTARTS   AGE
kube-dns-v11-osikn   4/4       Running   0          13m
</code></pre>

<p>And:</p>

<pre><code>kubectl logs kube-dns-v11-osikn kube2sky --namespace=kube-system
I0621 15:44:48.168080       1 kube2sky.go:462] Etcd server found: http://127.0.0.1:4001
I0621 15:44:49.170404       1 kube2sky.go:529] Using https://192.168.0.1:443 for kubernetes master
I0621 15:44:49.170422       1 kube2sky.go:530] Using kubernetes API &lt;nil&gt;
I0621 15:44:49.170823       1 kube2sky.go:598] Waiting for service: default/kubernetes
I0621 15:44:49.209691       1 kube2sky.go:660] Successfully added DNS record for Kubernetes service.
</code></pre>

<p><strike>"Using kubernetes API <code>&lt;nil&gt;</code>" is a problem, isn't it ?</strike>
edit: I forced kube-master-url in the pod to let kube2sky contacting the master.</p>

<pre><code>kubectl logs kube-dns-v11-osikn skydns --namespace=kube-system
2016/06/21 15:44:50 skydns: falling back to default configuration, could not read from etcd: 100: Key not found (/skydns/config) [10]
2016/06/21 15:44:50 skydns: ready for queries on cluster.local. for tcp://0.0.0.0:53 [rcache 0]
2016/06/21 15:44:50 skydns: ready for queries on cluster.local. for udp://0.0.0.0:53 [rcache 0]
</code></pre>

<p>Note this too:</p>

<pre><code>kubectl get pods --all-namespaces 
NAMESPACE     NAME                 READY     STATUS    RESTARTS   AGE
default       busybox              1/1       Running   0          17m
kube-system   kube-dns-v11-osikn   4/4       Running   0          18m
</code></pre>

<p>So I've got no problem with skydns.</p>

<p>I'm sure that the problem comes from kubelet, I've tried to remove /var/lib/kubelet and restart the entire cluster. I've tried to restart kubelete services before and after installing dns also. I changed docker configuration, removed "--dns" option afterwoard, and I've got the same behaviour: Docker + dns is ok, Kubelet gives a MissingClusterDNS error saying that kubelet has got no configured cluster dns.</p>

<p>So please... Help (one more time :) )</p>

<hr>

<p>EDITS:
- now kube2sky doesn't complain about <code>&lt;nil&gt;</code> api version forcing kube2sky option
- I can force nslookup to use my sky DNS:</p>

<pre><code>kubectl exec busybox -- nslookup kubernetes.default.svc.cluster.local 192.168.0.10
Server:    192.168.0.10
Address 1: 192.168.0.10

Name:      kubernetes.default.svc.cluster.local
Address 1: 192.168.0.1
</code></pre>

<p>But the "MissingClusterDNS" error remains on pod creation, as if kubelet doesn't the startup options "--cluster-dns" and "--cluster-domain"</p>

<p>@Brendan Burns:</p>

<pre><code>kubectl get services --namespace=kube-system
NAME       CLUSTER-IP     EXTERNAL-IP   PORT(S)         AGE
kube-dns   192.168.0.10   &lt;none&gt;        53/UDP,53/TCP   12m
</code></pre>

<p>I finally managed my problem... Shame on me (or not).</p>

<p>I've taken kubelet sources to understand what happends and now I found. </p>

<p>In the "kubelet" file, I set:</p>

<pre><code>KUBE_ARGS="--cluster-dns=10.10.0.10 --cluster-domain=cluster.local"
</code></pre>

<p>And the log I added in source says that "cluster-dns" option as this value:</p>

<pre><code>10.10.0.10 --cluster-domain=cluster.local
</code></pre>

<p>That's mainly because the config file is interpreted by SystemD as a "bash environment vars" so KUBE_ARGS is "one argument", and it's badly parsed by kubelet service.</p>

<p>The solution is to split variable in two and change kubelet.service file to use vars. After a call to <code>systemctl daemon-reload; systemctl restart kubelet</code> everything is ok.</p>

<p>I opened an issue here: <a href="https://github.com/kubernetes/kubernetes/issues/27722" rel="nofollow">https://github.com/kubernetes/kubernetes/issues/27722</a> where I explain that the comment in example config file is ambiguous and/or the arguments are not parsed as expected.</p>
