<p>I am building a CI/CD pipeline using <strong>Tekton</strong> on a bare metal Kubernetes Cluster. I have managed to cache the necessary images (Node &amp; Nginx) and the layers, but how can I cache the .cache / public folders created by <strong>Gatsby build</strong>? These folders are not present in the repo. If the build step does not find these folders in takes longer because it needs to create all images using Sharp.</p>
<p>The pipeline has a PVC attached. In the task it is called <em>source</em> (workspaces). To be more clear, how can I copy the Gatsby folders to this PVC after the build has finished and to the Kaniko container before the next build?</p>
<p><strong>The Tekton task has the following steps:</strong></p>
<ol>
<li>Use Kaniko warmer to cache Docker Images used in the Docker build</li>
<li>Create a timestamp so that &quot;RUN build&quot; is executed every time even if the files don't change because it runs a GraphQL query</li>
<li>Build and push image using Kaniko</li>
<li>&amp; 5. Export image digest used by next step in the pipeline</li>
</ol>
<pre class="lang-yaml prettyprint-override"><code>apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: build-docker-image
spec:
  params:
    - name: pathToDockerFile
      type: string
      description: The path to the dockerfile to build
      default: $(resources.inputs.source-repo.path)/Dockerfile
    - name: pathToContext
      type: string
      description: |
        The build context used by Kaniko
        (https://github.com/GoogleContainerTools/kaniko#kaniko-build-contexts)
      default: $(resources.inputs.source-repo.path)
  resources:
    inputs:
      - name: source-repo
        type: git
    outputs:
      - name: builtImage
        type: image
      - name: event-to-sink
        type: cloudEvent
  workspaces:
    # PVC
    - name: source
      description: |
        Folder to write docker image digest
  results:
    - name: IMAGE-DIGEST
      description: Digest of the image just built.
  steps:
    - name: kaniko-warmer
      image: gcr.io/kaniko-project/warmer
      workingDir: $(workspaces.source.path)
      args:
        - --cache-dir=$(workspaces.source.path)/cache
        - --image=node:14-alpine
        - --image=nginx:1.19.5
    - name: print-date-unix-timestamp
      image: bash:latest
      script: |
        #!/usr/bin/env bash
        date | tee $(params.pathToContext)/date
    - name: build-and-push
      workingDir: $(workspaces.source.path)
      image: gcr.io/kaniko-project/executor:v1.3.0
      env:
        - name: 'DOCKER_CONFIG'
          value: '/tekton/home/.docker/'
      command:
        - /kaniko/executor
      args:
        - --build-arg=CACHEBUST=$(params.pathToContext)/date
        - --dockerfile=$(params.pathToDockerFile)
        - --destination=$(resources.outputs.builtImage.url)
        - --context=$(params.pathToContext)
        - --cache=true
        - --cache-ttl=144h
        - --cache-dir=$(workspaces.source.path)/cache
        - --use-new-run
        - --snapshotMode=redo
        - --cache-repo=&lt;repo&gt;/kaniko-cache
        - --log-timestamp
      securityContext:
        runAsUser: 0
    - name: write-digest
      workingDir: $(workspaces.source.path)
      image: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/imagedigestexporter:v0.16.2
      command: ['/ko-app/imagedigestexporter']
      args:
        - -images=[{&quot;name&quot;:&quot;$(resources.outputs.builtImage.url)&quot;,&quot;type&quot;:&quot;image&quot;,&quot;url&quot;:&quot;$(resources.outputs.builtImage.url)&quot;,&quot;digest&quot;:&quot;&quot;,&quot;OutputImageDir&quot;:&quot;$(workspaces.source.path)/$(params.pathToContext)/image-digest&quot;}]
        - -terminationMessagePath=$(params.pathToContext)/image-digested
      securityContext:
        runAsUser: 0
    - name: digest-to-result
      workingDir: $(workspaces.source.path)
      image: docker.io/stedolan/jq@sha256:a61ed0bca213081b64be94c5e1b402ea58bc549f457c2682a86704dd55231e09
      script: |
        cat $(params.pathToContext)/image-digested | jq '.[0].value' -rj | tee /$(results.IMAGE-DIGEST.path)
</code></pre>
<p><strong>Dockerfile</strong></p>
<pre><code>FROM node:14-alpine as build
ARG CACHEBUST=1

RUN apk update \
  &amp;&amp; apk add \
  build-base \
  libtool \
  autoconf \
  automake \
  pkgconfig \
  nasm \
  yarn \
  libpng-dev libjpeg-turbo-dev giflib-dev tiff-dev \
  zlib-dev \
  python \
  &amp;&amp; rm -rf /var/cache/apk/*

EXPOSE 8000 9000

RUN yarn global add gatsby-cli

WORKDIR /usr/src/app
COPY ./package.json .
RUN yarn install
COPY . .
RUN yarn build &amp;&amp; echo $CACHEBUST

CMD [&quot;yarn&quot;, &quot;serve&quot;]

FROM nginx:1.19.5 as serve
EXPOSE 80
COPY --from=build /usr/src/app/public /usr/share/nginx/html
</code></pre>

<blockquote>
<p>how can I cache the .cache / public folders created by Gatsby build? These folders are not present in the repo.</p>
</blockquote>
<p>If Persistent Volumes is available on your cluster and these volumes is available from all nodes, you can use a PVC-backed workspace for cache.</p>
<p>A more generic solution that also works in a regional cluster (e.g. cloud) is to upload the cached folder to something, e.g. a Bucket (<a href="https://min.io/" rel="nofollow noreferrer">Minio</a>?) or potentially Redis? Then also need a Task that download this folder - potentially in parallel with <code>git clone</code> when starting a new <code>PipelineRun</code>. GitHub Actions has a similar solution with the <a href="https://github.com/actions/cache" rel="nofollow noreferrer">cache action</a>.</p>
<p>Example of a Task with two workspaces that copy a file from one workspace to the other:</p>
<pre><code>apiVersion: tekton.dev/v1beta1  
kind: Task  
metadata:   
  name: copy-between-workspaces 
spec:   
  workspaces:   
    - name: ws-a    
    - name: ws-b    
  steps:    
    - name: copy
      image: ubuntu 
      script: cp $(workspaces.ws-a.path)/myfile $(workspaces.ws-b.path)/myfile
</code></pre>
