🐍Configure Python environments

Navigating multiple development environments.

ZenML deployments often involve multiple environments. This guide helps you manage dependencies and configurations across these environments.

Here is a visual overview of the different environments:

Client Environment (or the Runner environment)

The client environment (sometimes known as the runner environment) is where the ZenML pipelines are compiled, i.e., where you call the pipeline function (typically in a run.py script). There are different types of client environments:

A local development environment

A CI runner in production.

A ZenML Pro runner.

A runner image orchestrated by the ZenML server to start pipelines.

In all the environments, you should use your preferred package manager (e.g., pip or poetry) to manage dependencies. Ensure you install the ZenML package and any required integrations.

The client environment typically follows these key steps when starting a pipeline:

Compiling an intermediate pipeline representation via the @pipeline function.

Creating or triggering pipeline and step build environments if running remotely.

Triggering a run in the orchestrator.

Please note that the @pipeline function in your code is only ever called in this environment. Therefore, any computational logic that is executed in the pipeline function needs to be relevant to this so-called compile time, rather than at execution time, which happens later.

ZenML Server Environment

The ZenML server environment is a FastAPI application managing pipelines and metadata. It includes the ZenML Dashboard and is accessed when you deploy ZenML. To manage dependencies, install them during ZenML deployment, but only if you have custom integrations, as most are built-in.

See also here for more on configuring the server environment.

Execution Environments