l Airflow server by running the following command:# Switch to the Python environment that has Airflow installed before running this command

airflow standalone

This command will start up an Airflow server on your local machine. During the startup, it will print a username and password which you can use to log in to the Airflow UI here.

We can now switch back the Python environment in which ZenML is installed and run a pipeline:

# Switch to the Python environment that has ZenML installed before running this command

python file_that_runs_a_zenml_pipeline.py

This call will produce a .zip file containing a representation of your ZenML pipeline for Airflow. The location of this .zip file will be in the logs of the command above. We now need to copy this file to the Airflow DAGs directory, from where the local Airflow server will load it and run your pipeline (It might take a few seconds until the pipeline shows up in the Airflow UI). To figure out the DAGs directory, we can run airflow config get-value core DAGS_FOLDER while having our Python environment with the Airflow installation active.

To make this process easier, we can configure our ZenML Airflow orchestrator to automatically copy the .zip file to this directory for us. To do so, run the following command:

# Switch to the Python environment that has ZenML installed before running this command

zenml orchestrator update --dag_output_dir=<AIRFLOW_DAG_DIRECTORY>

Now that we've set this up, running a pipeline in Airflow is as simple as just running the Python file:

# Switch to the Python environment that has ZenML installed before running this command

python file_that_runs_a_zenml_pipeline.py

When using the Airflow orchestrator with a remote deployment, you'll additionally need:

A remote ZenML server deployed to the cloud. See the deployment guide for more information.

A deployed Airflow server. See the deployment section for more information.

A remote artifact store as part of your stack.

A remote container registry as part of your stack.