<p>I have a Kubernetes cluster that takes jobs for processing. These jobs are defined as follows:</p>

<pre><code>apiVersion: batch/v1
kind: Job
metadata:
  name: process-item-014
  labels:
    jobgroup: JOB_XXX
spec:
  template:
    metadata:
      name: JOB_XXX
      labels:
        jobgroup: JOB_XXX
    spec:
      restartPolicy: OnFailure
      containers:
      - name: worker
        image: gcr.io/.../worker
        volumeMounts: 
        - mountPath: /workspace
          name: workspace
        resources:
          limits:
            cpu: 500m
            memory: 1Gi
          requests:
            cpu: 500m
            memory: 512Mi  
      volumes: 
        - name: workspace 
          hostPath:
            path: /tmp/client-workspace
</code></pre>

<p>Note that I'm trying to mount a folder in the host into the container (<code>workspace</code>). Note also the memory limits defined.
On my container, I download a number of files into <code>workspace</code>, some of them being pretty large (They are downloaded with gsutil from GCS, but don't think that's too important).</p>

<p>When the files I download exceed the memory limits, my code breaks with a "device out of space" error. This doesn't completely make sense, because I'm storing the files into a mount, that is backed by the host's storage, which is more than enough. It's also mentioned in the <a href="https://kubernetes.io/docs/tasks/configure-pod-container/assign-cpu-ram-container/#assigning-cpu-and-ram-resources-to-a-container" rel="nofollow noreferrer">docs</a> that <code>memory</code> limit's the ammount of RAM available for the container, not storage. Still, when I set the limit to <code>XGi</code>, it breaks after <code>XGi</code> download pretty consistently. </p>

<p>My container is based on ubuntu:14.04, running a shell script with a line like this:</p>

<pre><code>gsutil -m cp -r gs://some/cloud/location/*  /workspace/files
</code></pre>

<p>What am I doing wrong? Will definitely need to have some limits for my containers, so I can't just drop the limits.</p>

<p>The <code>/tmp</code> filesystem is often backed by <a href="https://en.wikipedia.org/wiki/Tmpfs" rel="nofollow noreferrer">tmpfs</a>, which stores files in memory rather than on disk. My guess is that is the case on your nodes, and the memory is being correctly charged to the container. Can you use an <a href="https://kubernetes.io/docs/concepts/storage/volumes/#emptydir" rel="nofollow noreferrer">emptydir</a> volume instead?</p>
