title: Airflow directories, read a file in a task
body: <p>I've discovered Airflow recently and I want to do a couple of simple examples to know how it works.
  So I've build a very simple DAG (example_airflow.py) like this</p>
  <pre><code>import airflow
  from airflow import DAG
  from airflow.operators.python_operator import PythonOperator
  from datetime import timedelta
  
  def first_task(**kwargs):
      a = 1+1
      print(a)
      kwargs['ti'].xcom_push(key='val', value=a)
  
  def second_task(ti):
      a = ti.xcom_pull(key='val', task_ids='first_task')
      b = a+a
      print(b)
      return b
  
  default_args = {
          'owner': 'airflow',    
          'start_date': airflow.utils.dates.days_ago(2),
          'retries': 1,
          'retry_delay': timedelta(minutes=5),
      }
  
  with DAG(dag_id=&quot;ejemplo_airflow&quot;,
          schedule_interval=&quot;@daily&quot;,
          default_args=default_args,
          catchup=False) as dag:
     
  
      first_task = PythonOperator(
          task_id=&quot;first_task&quot;,
          python_callable=first_task,
          provide_context=True)
  
      second_task = PythonOperator(
          task_id=&quot;second_task&quot;,
          python_callable=second_task,
          provide_context=True)
  
  first_task &gt;&gt; second_task
  </code></pre>
  <p>When I run <code>docker-compose up</code> and go to localhost everything works perfectly, and the second task prints the value 4. Now I tried something like this</p>
  <pre><code>import airflow
  from airflow import DAG
  from airflow.operators.python_operator import PythonOperator
  from datetime import timedelta
  
  def read_data(**kwargs):
      import pandas as pd
      df = pd.read_pickle(&quot;./dags/data/example.pkl&quot;)
      kwargs['ti'].xcom_push(key='df', value=df)
  
  def first_task(ti):
      df = ti.xcom_pull(key='df', task_ids='read_data')
      x = df.shape
      print(x)
      return x
  
  ...
  
  read_data &gt;&gt; first_task
  </code></pre>
  <p>And when I run <code>docker-compose up</code> it fails the task <code>read_data</code>. It says that 'No such file or directory' exists.</p>
  <p>This is my structure:</p>
  <pre><code>├───home
  │   ├───airflow
  │   │   ├───example
  │   │   ├──────dags
  │   │   ├────────────example_airflow.py
  │   │   ├────────────data
  │   │   ├───────────────example.pkl
  │   │   │──────logs
  │   │   │──────plugins
  │   │   ├───docker-compose.yaml
  </code></pre>
  <p>I'm running Airflow 2.3.2 in my local machine with Ubuntu 22.04. I tried to do a task with</p>
  <pre><code>def read_file(**kwargs):
      import pandas as pd
      import os
      AIRFLOW_HOME = os.getenv('AIRFLOW_HOME')
      file_df = pd.read_pickle(AIRFLOW_HOME + '/dags/data/example.pkl')
      kwargs['ti'].xcom_push(key='df', value=file_df)
  </code></pre>
  <p>and it doesn't work either. When I write a task like this</p>
  <pre><code>import os
  print(os.getcwd())
  </code></pre>
  <p>or with <code>AIRFLOW_HOME</code> it says that the directory is <code>/opt/***</code>.</p>
  <p>Can someone explain to me how the directories work in Airflow or how should I do these kind of tasks?</p>