<p>There are a few processes I'm struggling to wrap my brain around when it comes to multi-stage <code>Dockerfile</code>.</p>
<p>Using this as an example, I have a couple questions below it:</p>
<pre><code># Dockerfile
# Uses multi-stage builds requiring Docker 17.05 or higher
# See https://docs.docker.com/develop/develop-images/multistage-build/

# Creating a python base with shared environment variables
FROM python:3.8.1-slim as python-base
ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    PIP_NO_CACHE_DIR=off \
    PIP_DISABLE_PIP_VERSION_CHECK=on \
    PIP_DEFAULT_TIMEOUT=100 \
    POETRY_HOME=&quot;/opt/poetry&quot; \
    POETRY_VIRTUALENVS_IN_PROJECT=true \
    POETRY_NO_INTERACTION=1 \
    PYSETUP_PATH=&quot;/opt/pysetup&quot; \
    VENV_PATH=&quot;/opt/pysetup/.venv&quot;

ENV PATH=&quot;$POETRY_HOME/bin:$VENV_PATH/bin:$PATH&quot;


# builder-base is used to build dependencies
FROM python-base as builder-base
RUN apt-get update \
    &amp;&amp; apt-get install --no-install-recommends -y \
        curl \
        build-essential

# Install Poetry - respects $POETRY_VERSION &amp; $POETRY_HOME
ENV POETRY_VERSION=1.0.5
RUN curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python

# We copy our Python requirements here to cache them
# and install only runtime deps using poetry
WORKDIR $PYSETUP_PATH
COPY ./poetry.lock ./pyproject.toml ./
RUN poetry install --no-dev  # respects 


# 'development' stage installs all dev deps and can be used to develop code.
# For example using docker-compose to mount local volume under /app
FROM python-base as development
ENV FASTAPI_ENV=development

# Copying poetry and venv into image
COPY --from=builder-base $POETRY_HOME $POETRY_HOME
COPY --from=builder-base $PYSETUP_PATH $PYSETUP_PATH

# Copying in our entrypoint
COPY ./docker/docker-entrypoint.sh /docker-entrypoint.sh
RUN chmod +x /docker-entrypoint.sh

# venv already has runtime deps installed we get a quicker install
WORKDIR $PYSETUP_PATH
RUN poetry install

WORKDIR /app
COPY . .

EXPOSE 8000
ENTRYPOINT /docker-entrypoint.sh $0 $@
CMD [&quot;uvicorn&quot;, &quot;--reload&quot;, &quot;--host=0.0.0.0&quot;, &quot;--port=8000&quot;, &quot;main:app&quot;]


# 'lint' stage runs black and isort
# running in check mode means build will fail if any linting errors occur
FROM development AS lint
RUN black --config ./pyproject.toml --check app tests
RUN isort --settings-path ./pyproject.toml --recursive --check-only
CMD [&quot;tail&quot;, &quot;-f&quot;, &quot;/dev/null&quot;]


# 'test' stage runs our unit tests with pytest and
# coverage.  Build will fail if test coverage is under 95%
FROM development AS test
RUN coverage run --rcfile ./pyproject.toml -m pytest ./tests
RUN coverage report --fail-under 95


# 'production' stage uses the clean 'python-base' stage and copyies
# in only our runtime deps that were installed in the 'builder-base'
FROM python-base as production
ENV FASTAPI_ENV=production

COPY --from=builder-base $VENV_PATH $VENV_PATH
COPY ./docker/gunicorn_conf.py /gunicorn_conf.py

COPY ./docker/docker-entrypoint.sh /docker-entrypoint.sh
RUN chmod +x /docker-entrypoint.sh

COPY ./app /app
WORKDIR /app

ENTRYPOINT /docker-entrypoint.sh $0 $@
CMD [ &quot;gunicorn&quot;, &quot;--worker-class uvicorn.workers.UvicornWorker&quot;, &quot;--config /gunicorn_conf.py&quot;, &quot;main:app&quot;]
</code></pre>
<p>The questions I have:</p>
<ol>
<li><p>Are you <code>docker build ...</code> this entire image and then just <code>docker run ... --target=&lt;stage&gt;</code> to run a specific stage (<code>development</code>, <code>test</code>, <code>lint</code>, <code>production</code>, etc.) or are you only building and running the specific stages you need (e.g. <code>docker build ... -t test --target=test &amp;&amp; docker run test ...</code>)?</p>
<p>I want to say it isn't the former because you end up with a bloated image with build kits and what not... correct?</p>
</li>
<li><p>When it comes to local Kubernetes development (<code>minikube</code>, <code>skaffold</code>, <code>devspace</code>, etc.) and running unit tests, are you supposed referring to these stages in the <code>Dockerfile</code> (<code>devspace</code> Hooks or something) or using native test tools in the container (e.g. <code>npm test</code>, <code>./manage.py test</code>, etc.)?</p>
</li>
</ol>
<p>Thanks for clearing this questions up.</p>

cjones
<p>To answer from a less DevSpace-y persepctive and a more general Docker-y one (With no disrespect to Lukas!):</p>
<h2>Question 1</h2>
<h3>Breakdown</h3>
<blockquote>
<p>❌ Are you docker build ... this entire image and then just docker run ... --target= to run a specific stage</p>
</blockquote>
<p>You're close in your understanding and managed to outline the approach in your second part of the query:</p>
<blockquote>
<p>✅ or are you only building and running the specific stages you need (e.g. docker build ... -t test --target=test &amp;&amp; docker run test ...)?</p>
</blockquote>
<p>The <code>--target</code> option is not present in the <code>docker run</code> command, which can be seen when calling <code>docker run --help</code>.</p>
<blockquote>
<p>I want to say it isn't the former because you end up with a bloated image with build kits and what not... correct?</p>
</blockquote>
<p>Yes, it's impossible to do it the first way, as when <code>--target</code> is not specified, then only the final stage is incorporated into your image. This is a great benefit as it cuts down the final size of your container, while allowing you to use multiple directives.</p>
<h3>Details and Examples</h3>
<p>It is a flag that you can pass in at <em>build time</em> so that you can choose which layers to build specifically. It's a pretty helpful directive that can be used in a few different ways. There's a decent blog post <a href="https://www.docker.com/blog/advanced-dockerfiles-faster-builds-and-smaller-images-using-buildkit-and-multistage-builds/" rel="nofollow noreferrer">here</a> talking about the the new features that came out with multi-stage builds (<code>--target</code> is one of them)</p>
<p>For example, I've had a decent amount of success building projects in CI utilising different stages and targets, the following is pseudo-code, but hopefully the context is applied</p>
<pre><code># Dockerfile
FROM python as base

FROM base as dependencies

COPY requirements.txt .
RUN pip install -r requirements.txt

FROM dependencies as test

COPY src/ src/
COPY test/ test/

FROM dependencies as publish

COPY src/ src/

CMD ...
</code></pre>
<p>A Dockerfile like this would enable you to do something like this in your CI workflow, once again, pseudo-code-esque</p>
<pre><code>docker build . -t my-app:unit-test --target test
docker run my-app:unit-test pyunit ...
docker build . -t my-app:latest
docker push ...
</code></pre>
<p>In some scenarios, it can be quite advantageous to have this fine grained control over what gets built when, and it's quite the boon to be able to run those images that comprise of only a few stages without having built the entire app.</p>
<p>The key here, is that there's no expectation that you need to use <code>--target</code>, but it <em>can</em> be used to solve particular problems.</p>
<h2>Question 2</h2>
<blockquote>
<p>When it comes to local Kubernetes development (minikube, skaffold, devspace, etc.) and running unit tests, are you supposed referring to these stages in the Dockerfile (devspace Hooks or something) or using native test tools in the container (e.g. npm test, ./manage.py test, etc.)?</p>
</blockquote>
<p>Lukas covers a devspace specific approach very well, but ultimately you can test however you like. Using devspace to make it easier to run (and remember to run) tests certainly sounds like a good idea. Whatever tool you use to enable an easier workflow, will likely still use <code>npm test</code> etc under the hood.</p>
<p>If you wish to call <code>npm test</code> outside of a container that's fine, if you wish to call it in a container, that's also fine. The solution to your problem will always change depending on your landscape. CICD helps to standardise on external factors and provide a uniform means to ensure testing is performed, and deployments are auditable</p>
<p>Hope that helps in any way shape or form 👍</p>
