<p>I'm currently migrating a DAG from airflow version 1.10.10 to 2.0.0.</p>
<p>This DAG uses a custom python operator where, depending on the complexity of the task, it assigns resources dynamically.
The problem is that the import used in v1.10.10 (<strong>airflow.contrib.kubernetes.pod import Resources</strong>) no longer works. I read that for v2.0.0 I should use <strong>kubernetes.client.models.V1ResourceRequirements</strong>, but I need to build this resource object dynamically.
This might sound dumb, but I haven't been able to find the correct way to build this object.</p>
<p>For example, I've tried with</p>
<pre><code>            self.resources = k8s.V1ResourceRequirements(
                request_memory=get_k8s_resources_mapping(resource_request)['memory'],
                limit_memory=get_k8s_resources_mapping(resource_request)['memory_l'],
                request_cpu=get_k8s_resources_mapping(resource_request)['cpu'],
                limit_cpu=get_k8s_resources_mapping(resource_request)['cpu_l']
            )
</code></pre>
<p>or</p>
<pre><code>            self.resources = k8s.V1ResourceRequirements(
                requests={'cpu': get_k8s_resources_mapping(resource_request)['cpu'],
                          'memory': get_k8s_resources_mapping(resource_request)['memory']},
                limits={'cpu': get_k8s_resources_mapping(resource_request)['cpu_l'],
                        'memory': get_k8s_resources_mapping(resource_request)['memory_l']}
            )
</code></pre>
<p>(get_k8s_resources_mapping(resource_request)['xxxx'] just returns a value depending on the resource_request, like '2Gi' for memory or '2' for cpu)</p>
<p>But they don't seem to work. The task fails.</p>
<p>So, my question is, how would you go about correctly building a V1ResourceRequirements in Python?
And, how should it look in the executor_config attribute of the task instance? Something like this, maybe?</p>
<pre><code>'resources': {'limits': {'cpu': '1', 'memory': '512Mi'}, 'requests': {'cpu': '1', 'memory': '512Mi'}}
</code></pre>

<p>The proper syntax is for example:</p>
<pre><code>from kubernetes import client
from airflow.providers.cncf.kubernetes.operators.kubernetes_pod import KubernetesPodOperator

KubernetesPodOperator(
    ...,
    resources = client.V1ResourceRequirements(
        requests={&quot;cpu&quot;: &quot;1000m&quot;, &quot;memory&quot;: &quot;8G&quot;},
        limits={&quot;cpu&quot;: &quot;16000m&quot;, &quot;memory&quot;: &quot;128G&quot;}
    )
)
</code></pre>
<p>If you would like to generate it dynamically simply replace the values in requests/limits with function that returns the <a href="https://github.com/kubernetes-client/python/blob/master/kubernetes/docs/V1ResourceRequirements.md" rel="nofollow noreferrer">expected string</a>.</p>
