<p>I have deployed airflow with helm stable/airflow: <a href="https://github.com/helm/charts/tree/master/stable/airflow" rel="nofollow noreferrer">https://github.com/helm/charts/tree/master/stable/airflow</a></p>
<p>I have deployed it with celery executor and I have run a task that need to read a big table. I am getting this error in the Pod that run this task:</p>
<pre><code>The node was low on resource: memory. Container base was using 3642536Ki, which exceeds its request of 0.
I understand that I have exceed the memory of the node is running this pod.
</code></pre>
<p>I want to modify the limit of the memory and cpu each pod use in airflow. I can see that I can limit the resources of the workers.</p>
<p>My question is:</p>
<p>How can I specify the memory requests, the limit for the pods that are going to be created from the worker? Because I can only see how to set the resources from the workers but not the Pots that the workers create.</p>

<p>I think you may be conflating the Celery Executor with the <a href="http://airflow.apache.org/docs/1.10.1/kubernetes.html" rel="nofollow noreferrer">Kubernetes Executor</a>.  The Kubernetes Executor creates a new pod for every task instance.  With the Celery Executor you will get a preset number of worker pods to run your tasks.  For example, you would set the number of workers in advance <a href="https://github.com/helm/charts/blob/master/stable/airflow/values.yaml#L543" rel="nofollow noreferrer">here</a>.  In which case, setting <a href="https://github.com/helm/charts/blob/master/stable/airflow/values.yaml#L523-L536" rel="nofollow noreferrer">these values</a> should give your worker pods the resources you require.  Their README has good example of how you can leverage dynamic scaling if that's what you're after.</p>
<p><a href="https://i.stack.imgur.com/0FvdU.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/0FvdU.png" alt="enter image description here" /></a></p>
