<p>I am looking for a way to "write stream" some .mp4 video files -- as they are being generated by some python app -- to a google cloud storage bucket. The python app is containerised and deployed in GKE and currently executes fine as a web service. But the problem is that all the video files are locally generated and stored in a path (<code>tmp/processed</code>) inside the pod. </p>

<p>However, I want the video files to be written to files in a google's storage bucket named <code>my_bucket</code>. </p>

<p>I have read <strong>gcsfuse</strong> guidelines (<a href="https://github.com/maciekrb/gcs-fuse-sample" rel="noreferrer">https://github.com/maciekrb/gcs-fuse-sample</a>) on how to mount a bucket in Kubernetes pods and also read about <strong>boto</strong> (<a href="https://cloud.google.com/storage/docs/boto-plugin#streaming-transfers" rel="noreferrer">https://cloud.google.com/storage/docs/boto-plugin#streaming-transfers</a>) that is used to do the stream transfers to storage buckets. </p>

<p>To mount <code>my_bucket</code> in <code>tmp/processed</code>, I have added the following lines to my app's deployment file (YAML):</p>

<pre><code>        lifecycle:
          postStart:
            exec:
              command:
              - gcsfuse
              - -o
              - nonempty
              - my_bucket
              - tmp/processed
          preStop:
            exec:
              command:
              - fusermount
              - -u
              - tmp/processed/
        securityContext:
          capabilities:
            add:
            - SYS_ADMIN
</code></pre>

<p>I haven't used boto yet and thought maybe just mounting would be enough! But, my app gives me <strong>input/output error</strong> when trying to generate the video file. </p>

<p>Now my question is that do I need to use both <strong>gcsfuse</strong> and <strong>boto</strong>, or just mounting the bucket in my GKE pod is enough? And am I doing the mounting right?</p>

<hr>

<p><strong>UPDATE</strong>: I verified that I did the mount correctly using the following command:</p>

<p><code>kubectl exec -it [POD_NAME] bash</code></p>

<p>Problem solved!
I only had to mount my bucket within the pod and that was it. The mounting script (as written above in my question) was done correctly. But, the problem that caused the <code>input/output error</code> was due to my GKE cluster that had insufficient permissions. Basically, the cluster didn't have the permission to read/write to storage and a couple of other permissions were needed by the project. So, I created a new cluster using the following command:</p>

<pre><code>gcloud container clusters create [MY_CLUSTER_NAME] \
  --scopes=https://www.googleapis.com/auth/userinfo.email,cloud-platform,https://www.googleapis.com/auth/devstorage.read_write,storage-rw,trace,https://www.googleapis.com/auth/trace.append,https://www.googleapis.com/auth/servicecontrol,compute-rw,https://www.googleapis.com/auth/compute,https://www.googleapis.com/auth/service.management.readonly,https://www.googleapis.com/auth/taskqueue \
  --num-nodes 4 --zone "us-central1-c"
</code></pre>

<p>to be able to read/write from/to a storage bucket the cluster had to have the <code>https://www.googleapis.com/auth/devstorage.read_write</code> permission.</p>

<p>Also, that there was no need to use <strong>boto</strong> and mounting through <strong>gcsfuse</strong> was enough for me to be able to write stream video files to <code>my_bucket</code>.</p>
