<p>This is the workflow I'm trying to parallelize:</p>
<ol>
<li>docker build</li>
<li>docker run</li>
<li>docker inspect container -&gt; returns false</li>
<li>docker cp container:file /host/</li>
</ol>
<p>I was wanting to use docker-compose to do this on a single host, then transition to Kubernetes later so that I can orchestrate this on multiple hosts.</p>
<p>Should I create a bash script and have it RUN in the Dockerfile?</p>
<p>I'm looking for a solution that the community accepts as the best practice.</p>

NomNomNom
<p>In single-host-Docker land, you can try to arrange things so that <code>docker run</code> does everything you need it to.  Avoid <code>docker inspect</code> (which dumps out low-level diagnostics that usually aren't interesting) and <code>docker cp</code>.  Depending on how you build things, you could build the artifact in your Dockerfile, and copy it out</p>
<pre class="lang-sh prettyprint-override"><code>docker build -t the-image .
# use &quot;cat&quot; to get it out?
docker run --rm the-image \
  cat /app/file \            # from the container
  &gt; file                     # via the container's stdout, to the host
# use volumes to get it out?
docker run --rm -v $PWD:/host the-image \
  cp /app/file /host
</code></pre>
<p>Depending on what you're building, you might extend this further to pass both the inputs and outputs in volumes, so the image is just a toolchain.  For a minimal Go application, using the <a href="https://hub.docker.com/_/golang/" rel="nofollow noreferrer">Docker Hub <code>golang</code> image</a>, for example:</p>
<pre class="lang-sh prettyprint-override"><code># don't docker build anything, but run a container with the toolchain
docker run --rm \
  -v $PWD:/app \
  -w /app \
  golang:1.15 \
  go build -o the_app ./cmd/the_app
</code></pre>
<p>In this last setup the <code>-w</code> working directory is the bind-mounted <code>/app</code> directory, so <code>go build -o ./the_app</code> writes out to the host.</p>
<p>Since this setup is a little more oriented towards single-use containers (note the <code>docker run --rm</code> option) it's not a good match for Compose, which generally expects long-running server-type containers.</p>
<hr />
<p>This setup also will not translate well to Kubernetes.  Kubernetes isn't really good at sharing files between containers or with systems outside the cluster.  If you happen to have an NFS server you can use that, but there aren't native options; most of the <a href="https://kubernetes.io/docs/concepts/storage/volumes/#volume-types" rel="nofollow noreferrer">volume types</a> that it's straightforward to get are <code>ReadWriteOnce</code> volumes that can't be reused between multiple Kubernetes Pods (containers).</p>
<p>You could in principle write a Kubernetes Job that did a single compilation.  It can't run <code>docker build</code>, so the &quot;run&quot; step would have to do the actual building.  You can't <code>kubectl cp</code> out of a completed pod (see <em>e.g.</em> <a href="https://github.com/kubernetes/kubectl/issues/454" rel="nofollow noreferrer">kubernetes/kubectl#454</a>), so it needs to send its content somewhere specific when it's done.</p>
<p>A better high-level approach here would be to find or install some sort of network-accessible storage, especially to hold the results (an Artifactory server; object storage like AWS S3).  Rewrite your build sequence as a &quot;task&quot; that takes the locations of the inputs and outputs and runs the build, ignoring the local filesystem.  Set up a job queue like RabbitMQ, and inject the tasks into the queue.  Finally, run the builder-worker as a Kubernetes Deployment; it will build as many things in parallel as the number of <code>replicas:</code> in the deployment.</p>
