<p>I have a bucket folder in Google Cloud with about <code>47GB</code> of data in it. I start a new Kubernetes <code>StatefulSet</code> (in my Google Cloud Kubernetes cluster). The first thing that the container inside the <code>StatefulSet</code> does is to use <code>gsutil -m rsync -r gs://&lt;BUCKET_PATH&gt; &lt;LOCAL_MOUNT_PATH&gt;</code> to sync the bucket folder contents to a locally mounted folder, which corresponds to a Kubernetes Persistent Volume. The Persistent Volume Claim for this <code>StatefulSet</code> requests <code>125Gi</code> of storage and is only used for this <code>rsync</code>. But the <code>gsutil</code> sync eventually hits a wall where the pod runs out of disk space (space in the Persistent Volume) and <code>gsutil</code> throws an error: <code>[Errno 28] No space left on device</code>. This is weird, because I only need to copy <code>47GB</code> of data over from the bucket, but the Persistent Volume should have <code>125Gi</code> of storage available.</p>
<p>I can confirm the Persistent Volume Claim and the Persistent Volume have been provisioned with the appropriate sizes by using <code>kubectl get pvc</code> and <code>kubectl get pv</code>. If I run <code>df -h</code> inside the pod (<code>kubectl exec -it &lt;POD_NAME&gt; -- df -h</code>) I can see that the mounted path exists and that it has the expected size (<code>125Gi</code>). Using <code>df -h</code> during the sync I can see that it does indeed take up all the available space in the Persistent Volume when it finally hits <code>No space left on device</code>.</p>
<p>Further, if I provision a Persistent Volume of <code>200Gi</code> and retry the sync, it finishes successfully and <code>df -h</code> shows that the used space in the Persistent Volume is <code>47GB</code>, as expected (this is after <code>gsutil rsync</code> is completed).</p>
<p>So it seems that <code>gsutil rsync</code> uses far more space <em>while syncing</em> than I would expect. Why is this? Is there a way to change how <code>gsutil rsync</code> is done so that it doesn't require a larger Persistent Volume than necessary?</p>
<p>It should be noted that there are a lot of individual files, and that the pod is restarted about 8 times during the sync.</p>

Spencer
<p><code>rsync</code> will transfer contents to a temporary file in the target folder first. If it succeeds then it will rename the file to become the target file. If the transfer fails, the temporary file will be deleted. You could try adding <code>--inplace</code> flag to the command, according to the <a href="https://download.samba.org/pub/rsync/rsync.html" rel="nofollow noreferrer">link</a>: “This option changes how rsync transfers a file when its data needs to be updated: instead of the default method of creating a new copy of the file and moving it into place when it is complete, rsync instead writes the updated data directly to the destination file.”</p>
