<p>I have a docker image <code>felipeogutierrez/tpch-dbgen</code> that I build using <code>docker-compose</code> and I push it to <a href="https://hub.docker.com/repository/docker/felipeogutierrez/tpch-dbgen" rel="nofollow noreferrer">docker-hub</a> registry using <code>travis-CI</code>.</p>
<pre><code>version: &quot;3.7&quot;
services:
  other-images: ....
  tpch-dbgen:
    build: ../docker/tpch-dbgen
    image: felipeogutierrez/tpch-dbgen
    volumes:
      - tpch-dbgen-data:/opt/tpch-dbgen/data/
      - datarate:/tmp/
    stdin_open: true
</code></pre>
<p>and this is the <code>Dockerfile</code> to build this image:</p>
<pre><code>FROM gcc AS builder
RUN mkdir -p /opt
COPY ./generate-tpch-dbgen.sh /opt/generate-tpch-dbgen.sh
WORKDIR /opt
RUN chmod +x generate-tpch-dbgen.sh &amp;&amp; ./generate-tpch-dbgen.sh
</code></pre>
<p>In the end, this scripts creates a directory <code>/opt/tpch-dbgen/data/</code> with some files that I would like to read from another docker image that I am running on Kubernetes. Then I have a Flink image that I create to run into Kubernetes. This image starts 3 Flink Task Managers and one stream application that reads files from the image <code>tpch-dbgen-data</code>. I think that the right approach is to create a <code>PersistentVolumeClaim</code> so I can share the directory <code>/opt/tpch-dbgen/data/</code> from image <code>felipeogutierrez/tpch-dbgen</code> to my flink image in Kubernetes. So, first I have this file to create the <code>PersistentVolumeClaim</code>:</p>
<pre><code>apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: tpch-dbgen-data-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 200Mi
</code></pre>
<p>Then, I am creating an <code>initContainers</code> to launch the image <code>felipeogutierrez/tpch-dbgen</code> and after that launch my image <code>felipeogutierrez/explore-flink:1.11.1-scala_2.12</code>:</p>
<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: flink-taskmanager
spec:
  replicas: 3
  selector:
    matchLabels:
      app: flink
      component: taskmanager
  template:
    metadata:
      labels:
        app: flink
        component: taskmanager
    spec:
      initContainers:
      - name: tpch-dbgen
        image: felipeogutierrez/tpch-dbgen
        #imagePullPolicy: Always
        env:
        command: [&quot;ls&quot;]
        # command: ['sh', '-c', 'for i in 1 2 3; do echo &quot;job-1 `date`&quot; &amp;&amp; sleep 5s; done;', 'ls']
        volumeMounts:
        - name: tpch-dbgen-data
          mountPath: /opt/tpch-dbgen/data
      containers:
      - name: taskmanager
        image: felipeogutierrez/explore-flink:1.11.1-scala_2.12
        #imagePullPolicy: Always
        env:
        args: [&quot;taskmanager&quot;]
        ports:
        - containerPort: 6122
          name: rpc
        - containerPort: 6125
          name: query-state
        livenessProbe:
          tcpSocket:
            port: 6122
          initialDelaySeconds: 30
          periodSeconds: 60
        volumeMounts:
        - name: flink-config-volume
          mountPath: /opt/flink/conf/
        - name: tpch-dbgen-data
          mountPath: /opt/tpch-dbgen/data
        securityContext:
          runAsUser: 9999  # refers to user _flink_ from official flink image, change if necessary
      volumes:
      - name: flink-config-volume
        configMap:
          name: flink-config
          items:
          - key: flink-conf.yaml
            path: flink-conf.yaml
          - key: log4j-console.properties
            path: log4j-console.properties
      - name: tpch-dbgen-data
        persistentVolumeClaim:
          claimName: tpch-dbgen-data-pvc
</code></pre>
<p>The Flink stream application is starting but it cannot read the files on the directory <code>/opt/tpch-dbgen/data</code> of the image <code>felipeogutierrez/tpch-dbgen</code>. I am getting the error: <code>java.io.FileNotFoundException: /opt/tpch-dbgen/data/orders.tbl (No such file or directory)</code>. It is strange because when I try to go into the container <code>felipeogutierrez/tpch-dbgen</code> I can list the files. So I suppose there is something wrong on my Kubernetes configuration. Does anyone know to point what I am missing on the Kubernetes configuration files?</p>
<pre><code>$ docker run -i -t felipeogutierrez/tpch-dbgen /bin/bash
root@10c0944a95f8:/opt# pwd
/opt
root@10c0944a95f8:/opt# ls tpch-dbgen/data/
customer.tbl  dbgen  dists.dss  lineitem.tbl  nation.tbl  orders.tbl  part.tbl  partsupp.tbl  region.tbl  supplier.tbl
</code></pre>
<p>Also, when I list the logs of the container <code>tpch-dbgen</code> I can see the directory <code>tpch-dbgen</code> that I want to read. Although I cannot execute the command <code>command: [&quot;ls tpch-dbgen&quot;]</code> inside my Kubernetes config file.</p>
<pre><code>$ kubectl get pods
NAME                                 READY   STATUS    RESTARTS   AGE
flink-jobmanager-n9nws               1/1     Running   2          17m
flink-taskmanager-777cb5bf77-ncdl4   1/1     Running   0          4m54s
flink-taskmanager-777cb5bf77-npmrx   1/1     Running   0          4m54s
flink-taskmanager-777cb5bf77-zc2nw   1/1     Running   0          4m54s
$ kubectl logs flink-taskmanager-777cb5bf77-ncdl4 tpch-dbgen
generate-tpch-dbgen.sh
tpch-dbgen
</code></pre>

Felipe
<p>I got to run the <code>PersistentVolumeClaim</code> and share it between pods. Basically I had to use a <code>subPath</code> property which I learned from this answer <a href="https://stackoverflow.com/a/43404857/2096986">https://stackoverflow.com/a/43404857/2096986</a> and I am using a simple <code>Job</code> that I learned from this answer <a href="https://stackoverflow.com/a/64023672/2096986">https://stackoverflow.com/a/64023672/2096986</a>. The final results is below:</p>
<p>The Dockerfile:</p>
<pre><code>FROM gcc AS builder

RUN mkdir -p /opt
COPY ./generate-tpch-dbgen.sh /opt/generate-tpch-dbgen.sh
WORKDIR /opt
RUN chmod +x /opt/generate-tpch-dbgen.sh
ENTRYPOINT [&quot;/bin/sh&quot;,&quot;/opt/generate-tpch-dbgen.sh&quot;]
</code></pre>
<p>and the script <code>generate-tpch-dbgen.sh</code> has to have this line in the end <code>sleep infinity &amp; wait</code> to not finalize. The <code>PersistentVolumeClaim</code> is the same of the question. Then I create a Job with the <code>subPath</code> property.</p>
<pre><code>apiVersion: batch/v1
kind: Job
metadata:
  name: tpch-dbgen-job
spec:
  template:
    metadata:
      labels:
        app: flink
        component: tpch-dbgen
    spec:
      restartPolicy: OnFailure
      volumes:
        - name: tpch-dbgen-data
          persistentVolumeClaim:
            claimName: tpch-dbgen-data-pvc
      containers:
        - name: tpch-dbgen
          image: felipeogutierrez/tpch-dbgen
          imagePullPolicy: Always
          volumeMounts:
          - mountPath: /opt/tpch-dbgen/data
            name: tpch-dbgen-data
            subPath: data
</code></pre>
<p>and I use it on the other deployment also with the <code>subPath</code> property.</p>
<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: flink-taskmanager
spec:
  replicas: 3
  selector:
    matchLabels:
      app: flink
      component: taskmanager
  template:
    metadata:
      labels:
        app: flink
        component: taskmanager
    spec:
      volumes:
      - name: flink-config-volume
        configMap:
          name: flink-config
          items:
          - key: flink-conf.yaml
            path: flink-conf.yaml
          - key: log4j-console.properties
            path: log4j-console.properties
      - name: tpch-dbgen-data
        persistentVolumeClaim:
          claimName: tpch-dbgen-data-pvc
      containers:
      - name: taskmanager
        image: felipeogutierrez/explore-flink:1.11.1-scala_2.12
        imagePullPolicy: Always
        env:
        args: [&quot;taskmanager&quot;]
        ports:
        - containerPort: 6122
          name: rpc
        - containerPort: 6125
          name: query-state
        livenessProbe:
          tcpSocket:
            port: 6122
          initialDelaySeconds: 30
          periodSeconds: 60
        volumeMounts:
        - name: flink-config-volume
          mountPath: /opt/flink/conf/
        - name: tpch-dbgen-data
          mountPath: /opt/tpch-dbgen/data
          subPath: data
        securityContext:
          runAsUser: 9999  # refers to user _flink_ from official flink image, change if necessary
</code></pre>
