<p>I have a docker image <code>felipeogutierrez/tpch-dbgen</code> that I build using <code>docker-compose</code> and I push it to <a href="https://hub.docker.com/repository/docker/felipeogutierrez/tpch-dbgen" rel="nofollow noreferrer">docker-hub</a> registry using <code>travis-CI</code>.</p>
<pre><code>version: &quot;3.7&quot;
services:
  other-images: ....
  tpch-dbgen:
    build: ../docker/tpch-dbgen
    image: felipeogutierrez/tpch-dbgen
    volumes:
      - tpch-dbgen-data:/opt/tpch-dbgen/data/
      - datarate:/tmp/
    stdin_open: true
</code></pre>
<p>and this is the <code>Dockerfile</code> to build this image:</p>
<pre><code>FROM gcc AS builder
RUN mkdir -p /opt
COPY ./generate-tpch-dbgen.sh /opt/generate-tpch-dbgen.sh
WORKDIR /opt
RUN chmod +x generate-tpch-dbgen.sh &amp;&amp; ./generate-tpch-dbgen.sh
</code></pre>
<p>In the end, this scripts creates a directory <code>/opt/tpch-dbgen/data/</code> with some files that I would like to read from another docker image that I am running on Kubernetes. Then I have a Flink image that I create to run into Kubernetes. This image starts 3 Flink Task Managers and one stream application that reads files from the image <code>tpch-dbgen-data</code>. I think that the right approach is to create a <code>PersistentVolumeClaim</code> so I can share the directory <code>/opt/tpch-dbgen/data/</code> from image <code>felipeogutierrez/tpch-dbgen</code> to my flink image in Kubernetes. So, first I have this file to create the <code>PersistentVolumeClaim</code>:</p>
<pre><code>apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: tpch-dbgen-data-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 200Mi
</code></pre>
<p>Then, I am creating an <code>initContainers</code> to launch the image <code>felipeogutierrez/tpch-dbgen</code> and after that launch my image <code>felipeogutierrez/explore-flink:1.11.1-scala_2.12</code>:</p>
<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: flink-taskmanager
spec:
  replicas: 3
  selector:
    matchLabels:
      app: flink
      component: taskmanager
  template:
    metadata:
      labels:
        app: flink
        component: taskmanager
    spec:
      initContainers:
      - name: tpch-dbgen
        image: felipeogutierrez/tpch-dbgen
        #imagePullPolicy: Always
        env:
        command: [&quot;ls&quot;]
        # command: ['sh', '-c', 'for i in 1 2 3; do echo &quot;job-1 `date`&quot; &amp;&amp; sleep 5s; done;', 'ls']
        volumeMounts:
        - name: tpch-dbgen-data
          mountPath: /opt/tpch-dbgen/data
      containers:
      - name: taskmanager
        image: felipeogutierrez/explore-flink:1.11.1-scala_2.12
        #imagePullPolicy: Always
        env:
        args: [&quot;taskmanager&quot;]
        ports:
        - containerPort: 6122
          name: rpc
        - containerPort: 6125
          name: query-state
        livenessProbe:
          tcpSocket:
            port: 6122
          initialDelaySeconds: 30
          periodSeconds: 60
        volumeMounts:
        - name: flink-config-volume
          mountPath: /opt/flink/conf/
        - name: tpch-dbgen-data
          mountPath: /opt/tpch-dbgen/data
        securityContext:
          runAsUser: 9999  # refers to user _flink_ from official flink image, change if necessary
      volumes:
      - name: flink-config-volume
        configMap:
          name: flink-config
          items:
          - key: flink-conf.yaml
            path: flink-conf.yaml
          - key: log4j-console.properties
            path: log4j-console.properties
      - name: tpch-dbgen-data
        persistentVolumeClaim:
          claimName: tpch-dbgen-data-pvc
</code></pre>
<p>The Flink stream application is starting but it cannot read the files on the directory <code>/opt/tpch-dbgen/data</code> of the image <code>felipeogutierrez/tpch-dbgen</code>. I am getting the error: <code>java.io.FileNotFoundException: /opt/tpch-dbgen/data/orders.tbl (No such file or directory)</code>. It is strange because when I try to go into the container <code>felipeogutierrez/tpch-dbgen</code> I can list the files. So I suppose there is something wrong on my Kubernetes configuration. Does anyone know to point what I am missing on the Kubernetes configuration files?</p>
<pre><code>$ docker run -i -t felipeogutierrez/tpch-dbgen /bin/bash
root@10c0944a95f8:/opt# pwd
/opt
root@10c0944a95f8:/opt# ls tpch-dbgen/data/
customer.tbl  dbgen  dists.dss  lineitem.tbl  nation.tbl  orders.tbl  part.tbl  partsupp.tbl  region.tbl  supplier.tbl
</code></pre>
<p>Also, when I list the logs of the container <code>tpch-dbgen</code> I can see the directory <code>tpch-dbgen</code> that I want to read. Although I cannot execute the command <code>command: [&quot;ls tpch-dbgen&quot;]</code> inside my Kubernetes config file.</p>
<pre><code>$ kubectl get pods
NAME                                 READY   STATUS    RESTARTS   AGE
flink-jobmanager-n9nws               1/1     Running   2          17m
flink-taskmanager-777cb5bf77-ncdl4   1/1     Running   0          4m54s
flink-taskmanager-777cb5bf77-npmrx   1/1     Running   0          4m54s
flink-taskmanager-777cb5bf77-zc2nw   1/1     Running   0          4m54s
$ kubectl logs flink-taskmanager-777cb5bf77-ncdl4 tpch-dbgen
generate-tpch-dbgen.sh
tpch-dbgen
</code></pre>

Felipe
<p>Docker has an unusual feature where, under some specific circumstances, it will populate a newly created volume from the image.  You should not rely on this functionality, since it completely ignores updates in the underlying images and it doesn't work on Kubernetes.</p>
<p>In your Kubernetes setup, you create a new empty PersistentVolumeClaim, and then mount this over your actual data in both the init and main containers.  As with all Unix mounts, this hides the data that was previously in that directory.  Nothing causes data to get copied into that volume.  This works the same way as every other kind of mount, except the Docker named-volume mount: you'll see the same behavior if you change your Compose setup to do a host bind mount, or if you play around with your local development system using a USB drive as a &quot;volume&quot;.</p>
<p>You need to make your init container (or something else) explicitly copy data into the directory.  For example:</p>
<pre class="lang-yaml prettyprint-override"><code>initContainers:
- name: tpch-dbgen
  image: felipeogutierrez/tpch-dbgen
  command:
    - /bin/cp
    - -a
    - /opt/tpch-dbgen/data
    - /data
  volumeMounts:
    - name: tpch-dbgen-data
      mountPath: /data # NOT the same path as in the image
</code></pre>
<p>If the main process modifies these files in place, you can make the command be more intelligent, or write a script into your image that only copies the individual files in if they don't exist yet.</p>
<p>It could potentially make more sense to have your image generate the data files at startup time, rather than at image-build time.  That could look like:</p>
<pre class="lang-sh prettyprint-override"><code>FROM gcc
COPY ./generate-tpch-dbgen.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/generate-tpch-dbgen.sh
CMD [&quot;generate-tpch-dbgen.sh&quot;]
</code></pre>
<p>Then in your init container, you can run the default command (the generate script) with the working directory set to the volume directory</p>
<pre class="lang-yaml prettyprint-override"><code>initContainers:
- name: tpch-dbgen
  image: felipeogutierrez/tpch-dbgen
  volumeMounts:
    - name: tpch-dbgen-data
      mountPath: /opt/tpch-dbgen/data # or anywhere really
  workingDir: /opt/tpch-dbgen/data    # matching mountPath
</code></pre>
