<p>I’ve been asked to configure a ubuntu 18.04 server with docker for multiple users.</p>
<p>Purpose:
We have multiple testers who write test cases. But our laptops aren’t fast enough to build the project and run tescases in docker environment.
We already have a jenkins server.But we need to build/test our code BEFORE push to git.</p>
<p>I’ve been given a high end ubuntu 18.04 server.
I have to configure the server where all our testers can run/debug our testcases on isolated environments.</p>
<p>When testers push there changes to remote servers project should build and run on isolated environments. Multiple users can work on same project but one testers builds must NOT affect another one.</p>
<p>I already installed Docker and tried with only changing docker-compose.yml and adding different networks (using multiple accounts of course). But it was very painful.</p>
<p>I need to have multiple selenoid servers(for different users),different allure reports with docker , Need the ability to build and run tests using our docker-compose files and need the ability to run the actual project on different ports so we can go through the system while writing test cases.</p>
<p>Is it possible to configure an environment without changing project docker-compose.yml ?
Whats the approach I should take ?</p>

<p>You can use Docker in Docker (<a href="https://hub.docker.com/_/docker" rel="noreferrer"><code>docker:dind</code></a> image) to run multiple instances of Docker daemon on the same host, and have each tester use a different <code>DOCKER_HOST</code> to run their Compose stack. Each app instance will be deployed on a separate Docker daemon and isolated without requiring any change in <code>docker-compose.yml</code>.</p>
<p>Docker in Docker can be used to run a Docker daemon from another Docker daemon. (Docker daemon is the process actually managing your container when using <code>docker</code>). See <a href="https://docs.docker.com/get-started/overview/#docker-architecture" rel="noreferrer">Docker architecture</a> and <a href="https://www.docker.com/blog/docker-can-now-run-within-docker/" rel="noreferrer">DinD original blogpost</a> for details.</p>
<hr />
<h1>Example: run 2 Docker daemons exposing the app port</h1>
<p>Let's Consider 2 testers with this <code>docker-compose.yml</code>:</p>
<pre><code>version: 3
services:
  app:
    image: my/app:latest
    ports:
      - 8080:80
</code></pre>
<ol>
<li>Run 2 instances of Docker Daemon exposing Daemon port <strong>and any port that will be exposed by Docker Compose</strong> (see below why)</li>
</ol>
<pre><code># Run docker dind and map port 23751 on localhost
# Expose Daemon 8080 on 8081 (port that will be used by Tester1)
# privileged is required to run dind (see dind-rootless exists but is experimental) 
# DOCKER_TLS_CERTDIR=&quot;&quot; is to deploy an unsecure Daemon
# it's easier to use but should only be used for testing/dev purposes
docker run -d \
  -p 23751:2375 \
  -p 8081:8080 \
  --privileged \
  --name dockerd-tester1 \
  -e DOCKER_TLS_CERTDIR=&quot;&quot; 
  docker:dind

# Second Daemon using port 23752
docker run -d \
  -p 23752:2375 \
  -p 8082:8080 \
  --privileged \
  --name dockerd-tester2 \
  -e DOCKER_TLS_CERTDIR=&quot;&quot; 
  docker:dind
</code></pre>
<ol start="2">
<li>Each tester can run their own stack on their Docker daemon by setting <code>DOCKER_HOST</code> env var:</li>
</ol>
<pre><code># Tester 1 shell
# use dockerd-tester1 daemon on port 23751
export DOCKER_HOST=tcp://localhost:23751

# run our stack
docker-compose up -d
</code></pre>
<p>Same for Tester 2 on <code>dockerd-tester2</code> port:</p>
<pre><code># Tester 2 shell
export DOCKER_HOST=tcp://localhost:23752
docker-compose up -d
</code></pre>
<ol start="3">
<li>Interacting with Tester 1 and 2's stacks</li>
</ol>
<blockquote>
<p>Need the ability to build and run tests using our docker-compose files and need the ability to run the actual project on different ports</p>
</blockquote>
<p>The exposed ports for each testers <strong>will be exposed on the Docker daemon host</strong> and reachable via <code>http://$DOCKER_HOST:$APP_PORT</code> instead of <code>localhost:$APP_PORT</code> (that's why we also exposed app port on each Daemon).</p>
<p>Considering our <code>docker-compose.yml</code>, testers will be able to access application such as:</p>
<pre><code># Tester 1
# port 8081 is linked to port 8080 of Docker daemon running our app container
# itself redirect on port 8080
# in short: 8081 -&gt; 8080 -&gt; 80
curl localhost:8081

# Tester 2
# 8082 -&gt; 8080 -&gt; 80
curl localhost:8082
</code></pre>
<p>Our deployment will look like this</p>
<p><img src="https://i.stack.imgur.com/Pe3mz.png" alt="enter image description here" /></p>
<hr />
<h1>Alternative without exposing ports, using Docker daemon IP directly</h1>
<p>Similar to the first example, you can also interact with the deployed app by using Docker daemon IP directly:</p>
<pre><code># Run daemon without exposing ports
docker run -d \
  --privileged \
  --name dockerd-tester1 \
  -e DOCKER_TLS_CERTDIR=&quot;&quot; 
  docker:dind

# Retrieve daemon IP
docker inspect --format '{{ .NetworkSettings.IPAddress }}' dockerd-tester1
# output like 172.17.0.2

# use it!
export DOCKER_HOST=172.17.0.2
docker-compose up -d

# our app port are exposed on Daemon
curl 172.17.0.2:8080
</code></pre>
<p>We contacted directly our Daemon via its IP instead of exposing its port on localhost.</p>
<hr />
<p>You can even define your Docker daemons with static IPs in a <code>docker-compose.yml</code> such as:</p>
<pre><code>version: &quot;3&quot;

services:
  dockerd-tester1:
    image: docker:dind
    privileged: true
    environment:
      DOCKER_TLS_CERTDIR: &quot;&quot;
    networks:
      dind-net:
        # static IP to set as DOCKER_HOST
        ipv4_address: 10.5.0.6

  # same for dockerd-tester2
  # ...

networks:
  dind-net:
    driver: bridge
    ipam:
     config:
       - subnet: 10.5.0.0/16
</code></pre>
<p>And then</p>
<pre><code>export DOCKER_HOST=10.5.0.6
# ...
</code></pre>
<hr />
<p>Notes:</p>
<ul>
<li>This may have some performance impact depending on the machine on which Daemons are deployed</li>
<li>You can use <code>dind-rootless</code> instead of <code>dind</code> to avoid using <code>--privileged</code> flags</li>
<li>It's better to avoid <code>DOCKER_TLS_CERTDIR: &quot;&quot;</code> for security reasons, see <a href="https://hub.docker.com/_/docker" rel="noreferrer"><code>TLS</code> instruction on <code>docker</code> image</a> for detailed usage of TLS</li>
</ul>
