title: Can I persist yarn install cache during docker build using RUN --mount=type=bind?
body: <p>I'm trying to generate and re-use a <code>yarn install</code> cache when building a Docker image using Docker BuildKit. The yarn cache is persisted in the directory <code>.yarn/cache</code> and should never be included in the final image (<code>.yarn/cache</code> is relative to the build context root). The <code>.yarn/cache</code> directory should be shared among multiple builds in order to always start from a warm cache and have a fast <code>yarn install</code> command (even when we have a cache miss due to a change in <code>package.json</code>). If we could have access to <code>.yarn/cache</code> content after <code>docker build</code> ends, will be easy to share between multiple builds, for example uploading it to an Amazon S3 or GCS bucket.</p>
  <p>I've considered two options:</p>
  <ol>
  <li><code>RUN --mount=type=bind</code></li>
  <li><code>RUN --mount=type=cache</code></li>
  </ol>
  <p>Described below why either of the two methods don't work.</p>
  <h2>(1) RUN --mount=type=bind</h2>
  <p>The (simplified) Dockerfile looks like this:</p>
  <pre><code>ENV YARN_CACHE_FOLDER &quot;.yarn/cache&quot;
  COPY package.json yarn.lock ./
  RUN --mount=type=bind,source=.yarn/cache,target=.yarn/cache,rw yarn install --frozen-lockfile
  </code></pre>
  <p>Unfortunately no data is present in <code>.yarn/cache</code> directory after <code>docker build</code> command ends.</p>
  <p>The reason that no data is persisted is described in the <code>rw</code> option <a href="https://hub.docker.com/r/docker/dockerfile" rel="nofollow noreferrer">documentation</a>: <code>Allow writes on the mount. Written data will be discarded</code>. If the written data is discarded, what's a working method for generating the cache the first time?</p>
  <h2>(2) <code>RUN --mount=type=cache</code></h2>
  <p>Alternatively I considered using <code>RUN --mount=type=cache</code>. Unfortunately there doesn't seem to be an easy way of persisting the cache in a local directory of the build host for being easily saved to an Amazon S3 or GCS bucket. If the cache is not persisted, we can't use it across different Cloud Builds if the Docker daemon state is not shared between them.</p>
  <p>To say it in another way: what is the best method for sharing a cache directory between different <code>docker build</code> that are running on different machines, without including this cache in the image? Is there any other way I'm missing here?</p>
  <ol>
  <li><code>RUN --mount=type=bind</code>: allow to mount a directory as if it was local, but effectively doesn't allow to write to that directory, so I can't generate the cache on the first run.</li>
  <li><code>RUN --mount=type=cache</code>: allow to share the cache between multiple builds on the same machine, but if we're running multiple different <code>docker build</code> (on different machines) it won't help because the cache is always empty.</li>
  </ol>