<p>I'm trying to create a kubernetes cluster following the document at: <a href="https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/" rel="noreferrer">https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/</a></p>

<p>First I have installed kubeadm with docker image on Coreos (1520.9.0) inside VirtualBox with Vagrant:</p>

<pre><code>docker run -it \
    -v /etc:/rootfs/etc \
    -v /opt:/rootfs/opt \
    -v /usr/bin:/rootfs/usr/bin \
    -e K8S_VERSION=v1.8.4 \
    -e CNI_RELEASE=v0.6.0 \
    xakra/kubeadm-installer:0.4.7 coreos
</code></pre>

<p>This was my kubeadm init:</p>

<p><code>kubeadm init --pod-network-cidr=10.244.0.0/16</code></p>

<p>When run the command:</p>

<pre><code>kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.9.1/Documentation/kube-flannel.yml
</code></pre>

<p>It returns:</p>

<pre><code>clusterrole "flannel" configured
clusterrolebinding "flannel" configured
serviceaccount "flannel" configured
configmap "kube-flannel-cfg" configured
daemonset "kube-flannel-ds" configured
</code></pre>

<p>But if I check "kubectl get pods --all-namespaces"</p>

<p>It returns:</p>

<pre><code>NAMESPACE     NAME                              READY     STATUS             RESTARTS   AGE
kube-system   etcd-coreos1                      1/1       Running            0          18m
kube-system   kube-apiserver-coreos1            1/1       Running            0          18m
kube-system   kube-controller-manager-coreos1   0/1       CrashLoopBackOff   8          19m
kube-system   kube-scheduler-coreos1            1/1       Running            0          18m
</code></pre>

<p>With <code>journalctl -f -u kubelet</code> I can see this error: <code>Unable to update cni config: No networks found in /etc/cni/net.d</code></p>

<p>I suspect that something was wrong with the command <code>kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.9.1/Documentation/kube-flannel.yml</code></p>

<p>Is there a way to know why this command doesn't work? Can I get some logs from anywhere?</p>

<p>Just tonight I used <a href="https://github.com/kubernetes-incubator/kubespray#readme" rel="noreferrer">kubespray</a> to provision a vagrant cluster, on CoreOS, using flannel (vxlan), and I was also mystified about how flannel could be a Pod inside Kubernetes</p>

<p>It turns out, <a href="https://github.com/kubernetes-incubator/kubespray/blob/79417e07ca4033d162eb94d8c66a82bf8f44f9ce/roles/network_plugin/flannel/templates/cni-flannel.yml.j2#L88-L89" rel="noreferrer">as seen here</a>, that they are using <a href="https://github.com/coreos/flannel-cni#readme" rel="noreferrer">flannel-cni</a> image <a href="https://github.com/kubernetes-incubator/kubespray/blob/79417e07ca4033d162eb94d8c66a82bf8f44f9ce/roles/download/defaults/main.yml#L66-L67" rel="noreferrer">from quay.io</a> to write out CNI files using a flannel side-car plus hostDir volume-mounts; it outputs <code>cni-conf.json</code> (that configures CNI to use flannel), and then <code>net-conf.json</code> (that configures the subnet and backend used by flannel).</p>

<p>I hope the jinja2 mustache syntax doesn't obfuscate the answer, but I found it very interesting to see how the Kubernetes folks chose to do it "for real" to compare and contrast against the example <code>DaemonSet</code> given in the flannel-cni README. I guess that's the long way of saying: try the descriptors in the flannel-cni README, then if it doesn't work see if they differ in some way from the known-working kubespray setup</p>

<p><em>update:</em> as a concrete example, observe that <a href="https://github.com/coreos/flannel/blob/v0.9.1/Documentation/kube-flannel.yml#L111" rel="noreferrer">the Documentation yaml</a> doesn't include the <a href="https://github.com/kubernetes-incubator/kubespray/blob/79417e07ca4033d162eb94d8c66a82bf8f44f9ce/roles/network_plugin/flannel/templates/cni-flannel.yml.j2#L68" rel="noreferrer"><code>--iface=</code></a> switch, and if your Vagrant setup is using both NAT and "private_network" then it likely means flannel is binding to <code>eth0</code> (the NAT one) and not <code>eth1</code> with a more static IP. I saw that caveat mentioned in the docs, but can't immediately recall where in order to cite it</p>

<p><em>update 2</em></p>

<blockquote>
  <p>Is there a way to know why this command doesn't work? Can I get some logs from anywhere?</p>
</blockquote>

<p>One may almost always access the logs of a Pod (even a statically defined one such as <code>kube-controller-manager-coreos1</code>) in the same manner: <code>kubectl --namespace=kube-system logs kube-controller-manager-coreos1</code>, and in the CrashLoopBackOff circumstance, adding in the <code>-p</code> for "-p"revious will show the logs from the most recent crash (but only for a few seconds, not indefinitely), and occasionally <code>kubectl --namespace=kube-system describe pod kube-controller-manager-coreos1</code> will show helpful information in either the Events section at the bottom, or in the "Status" block near the top if it was Terminated for cause</p>

<p>In the case of a very bad failure, such as the apiserver failing to come up (and thus <code>kubectl logs</code> won't do anything), then ssh-ing to the Node and using a mixture of <code>journalctl -u kubelet.service --no-pager --lines=150</code> and <code>docker logs ${the_sha_or_name}</code> to try and see any error text. You will almost certainly need <code>docker ps -a</code> in the latter case to find the exited container's sha or name, but that same "only for a few seconds" applies, too, as dead containers will be pruned after some time.</p>

<p>In the case of vagrant, one can ssh into the VM in one of several ways:</p>

<ul>
<li><code>vagrant ssh coreos1</code></li>
<li><code>vagrant ssh-config &gt; ssh-config &amp;&amp; ssh -F ssh-config coreos1</code></li>
<li>or if it has a "private_network" address, such as 192.168.99.101 or such, then you can usually <code>ssh -i ~/.vagrant.d/insecure_private_key core@192.168.99.101</code> but one of the first two are almost always more convenient</li>
</ul>
