Specify pip dependencies and apt packages

The configuration for specifying pip and apt dependencies only works in the remote pipeline case, and is disregarded for local pipelines (i.e. pipelines that run locally without having to build a Docker image).

When a pipeline is run with a remote orchestrator a Dockerfile is dynamically generated at runtime. It is then used to build the Docker image using the image builder component of your stack.

By default, ZenML automatically installs all packages required by your active ZenML stack. However, you can specify additional packages to be installed in various ways:

Install all the packages in your local Python environment (This will use the pip or poetry package manager to get a list of your local packages):

# or use "poetry_export"

docker_settings = DockerSettings(replicate_local_python_environment="pip_freeze")

@pipeline(settings={"docker": docker_settings})

def my_pipeline(...):

...

If required, a custom command can be provided. This command must output a list of requirements following the format of the requirements file:

docker_settings = DockerSettings(replicate_local_python_environment=[

"poetry",

"export",

"--extras=train",

"--format=requirements.txt"

])

@pipeline(settings={"docker": docker_settings})

def my_pipeline(...):

...

Specify a list of requirements in code:Copydocker_settings = DockerSettings(requirements=["torch==1.12.0", "torchvision"])

@pipeline(settings={"docker": docker_settings})
def my_pipeline(...):
    ...

Specify a requirements file:Copydocker_settings = DockerSettings(requirements="/path/to/requirements.txt")

@pipeline(settings={"docker": docker_settings})
def my_pipeline(...):
    ...

Specify a list of ZenML integrations that you're using in your pipeline:

from zenml.integrations.constants import PYTORCH, EVIDENTLY

docker_settings = DockerSettings(required_integrations=[PYTORCH, EVIDENTLY])

@pipeline(settings={"docker": docker_settings})

def my_pipeline(...):

...