<p>I intend to test a non-trivial Kubernetes setup as part of CI and wish to run the full system before CD. I cannot run <code>--privileged</code> containers and am running the docker container as a sibling to the host using <code>docker run -v /var/run/docker.sock:/var/run/docker.sock</code></p>
<p>The basic docker setup seems to be working on the <em><strong>container</strong></em>:</p>
<pre><code>linuxbrew@03091f71a10b:~$ docker run hello-world

Hello from Docker!
This message shows that your installation appears to be working correctly.
</code></pre>
<p>However, minikube fails to start <em><strong>inside the docker container</strong></em>, reporting connection issues:</p>
<pre><code>linuxbrew@03091f71a10b:~$ minikube start --alsologtostderr -v=7
I1029 15:07:41.274378    2183 out.go:298] Setting OutFile to fd 1 ...
I1029 15:07:41.274538    2183 out.go:345] TERM=xterm,COLORTERM=, which probably does not support color
...
...
...
I1029 15:20:27.040213     197 main.go:130] libmachine: Using SSH client type: native
I1029 15:20:27.040541     197 main.go:130] libmachine: &amp;{{{&lt;nil&gt; 0 [] [] []} docker [0x7a1e20] 0x7a4f00 &lt;nil&gt;  [] 0s} 127.0.0.1 49350 &lt;nil&gt; &lt;nil&gt;}
I1029 15:20:27.040593     197 main.go:130] libmachine: About to run SSH command:
sudo hostname minikube &amp;&amp; echo &quot;minikube&quot; | sudo tee /etc/hostname
I1029 15:20:27.040992     197 main.go:130] libmachine: Error dialing TCP: dial tcp 127.0.0.1:49350: connect: connection refused                                                  
</code></pre>
<p>This is despite the network being linked and the port being properly forwarded:</p>
<pre><code>linuxbrew@51fbce78731e:~$ docker container ls
CONTAINER ID   IMAGE                                 COMMAND                  CREATED         STATUS         PORTS                                                                                                                                  NAMES
93c35cec7e6f   gcr.io/k8s-minikube/kicbase:v0.0.27   &quot;/usr/local/bin/entr…&quot;   2 minutes ago   Up 2 minutes   127.0.0.1:49350-&gt;22/tcp, 127.0.0.1:49351-&gt;2376/tcp, 127.0.0.1:49348-&gt;5000/tcp, 127.0.0.1:49349-&gt;8443/tcp, 127.0.0.1:49347-&gt;32443/tcp   minikube
51fbce78731e   7f7ba6fd30dd                          &quot;/bin/bash&quot;              8 minutes ago   Up 8 minutes                                                                                                                                          bpt-ci
linuxbrew@51fbce78731e:~$ docker network ls
NETWORK ID     NAME       DRIVER    SCOPE
1e800987d562   bridge     bridge    local
aa6b2909aa87   host       host      local
d4db150f928b   kind       bridge    local
a781cb9345f4   minikube   bridge    local
0a8c35a505fb   none       null      local
linuxbrew@51fbce78731e:~$ docker network connect a781cb9345f4 93c35cec7e6f
Error response from daemon: endpoint with name minikube already exists in network minikube
</code></pre>
<p>The minikube container seems to be alive and well when trying to <code>curl</code> <em><strong>from the host</strong></em> and even <code>ssh</code>is responding:</p>
<pre><code>mastercook@linuxkitchen:~$ curl https://127.0.0.1:49350
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 127.0.0.1:49350 

mastercook@linuxkitchen:~$ ssh root@127.0.0.1 -p 49350
The authenticity of host '[127.0.0.1]:49350 ([127.0.0.1]:49350)' can't be established.
ED25519 key fingerprint is SHA256:0E41lExrrezFK1QXULaGHgk9gMM7uCQpLbNPVQcR2Ec.
This key is not known by any other names
</code></pre>
<p>What am I missing and how can I make minikube properly discover the correctly working minikube container?</p>

<p>Because <code>minikube</code> does not complete the cluster creation, running Kubernetes in a (sibling) Docker container favours <code>kind</code>.</p>
<p>Given that the (sibling) container does not know enough about its setup, the networking connections are a bit flawed. Specifically, a loopback IP is selected by <code>kind</code> (and minikube) upon cluster creation even though the actual container sits on a different IP in the host docker.</p>
<p>To correct the networking, the (sibling) container needs to be connected to the network actually hosting the Kubernetes image. To accomplish this, the procedure is illustrated below:</p>
<ol>
<li>Create a kubernetes cluster:</li>
</ol>
<pre><code>linuxbrew@324ba0f819d7:~$ kind create cluster --name acluster
Creating cluster &quot;acluster&quot; ...
 ✓ Ensuring node image (kindest/node:v1.21.1) 🖼
 ✓ Preparing nodes 📦  
 ✓ Writing configuration 📜 
 ✓ Starting control-plane 🕹️ 
 ✓ Installing CNI 🔌 
 ✓ Installing StorageClass 💾 
Set kubectl context to &quot;kind-acluster&quot;
You can now use your cluster with:

kubectl cluster-info --context kind-acluster

Thanks for using kind! 😊
</code></pre>
<ol start="2">
<li>Verify if the cluster is accessible:</li>
</ol>
<pre><code>linuxbrew@324ba0f819d7:~$ kubectl cluster-info --context kind-acluster

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
The connection to the server 127.0.0.1:36779 was refused - did you specify the right host or port?
</code></pre>
<p>3.) Since the cluster cannot be reached, retrieve the control planes master IP. Note the &quot;-control-plane&quot; addition to the cluster name:</p>
<pre><code>linuxbrew@324ba0f819d7:~$ export MASTER_IP=$(docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' acluster-control-plane)
</code></pre>
<p>4.) Update the kube config with the actual master IP:</p>
<pre><code>linuxbrew@324ba0f819d7:~$ sed -i &quot;s/^    server:.*/    server: https:\/\/$MASTER_IP:6443/&quot; $HOME/.kube/config
</code></pre>
<p>5.) This IP is still not accessible by the (sibling) container and to connect the container with the correct network retrieve the docker network ID:</p>
<pre><code>linuxbrew@324ba0f819d7:~$ export MASTER_NET=$(docker inspect --format='{{range .NetworkSettings.Networks}}{{.NetworkID}}{{end}}' acluster-control-plane)
</code></pre>
<p>6.) Finally connect the (sibling) container ID (which should be stored in the <code>$HOSTNAME</code> environment variable) with the cluster docker network:</p>
<pre><code>linuxbrew@324ba0f819d7:~$ docker network connect $MASTER_NET $HOSTNAME
</code></pre>
<p>7.) Verify whether the control plane accessible after the changes:</p>
<pre><code>linuxbrew@324ba0f819d7:~$ kubectl cluster-info --context kind-acluster
Kubernetes control plane is running at https://172.18.0.4:6443
CoreDNS is running at https://172.18.0.4:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
</code></pre>
<p>If <code>kubectl</code> returns Kubernetes control plane and CoreDNS URL, as shown in the last step above, the configuration has succeeded.</p>
