on install kubeflow

Docker installed and running.kubectl installed (optional, see below)

If you are using a single-tenant Kubeflow installed in a Kubernetes cluster managed by a cloud provider like AWS, GCP or Azure, it is recommended that you set up a Service Connector and use it to connect ZenML Stack Components to the remote Kubernetes cluster. This guarantees that your Stack is fully portable on other environments and your pipelines are fully reproducible.

When using the Kubeflow orchestrator locally, you'll additionally need:

K3D installed to spin up a local Kubernetes cluster.

Terraform installed to set up the Kubernetes cluster with various deployments.

MLStacks installed to handle the deployment

To run the pipeline on a local Kubeflow Pipelines deployment, you can use the ZenML mlstacks package to spin up a local Kubernetes cluster and install Kubeflow Pipelines on it.

To deploy the stack, run the following commands:

# Deploy the stack using the ZenML CLI:

zenml stack deploy k3d-modular -o kubeflow -a minio --provider k3d

zenml stack set k3d-modular

# Get the Kubeflow Pipelines UI endpoint

kubectl get ingress -n kubeflow  -o jsonpath='{.items[0].spec.rules[0].host}'

You can read more about mlstacks on our dedicated documentation page here.

The local Kubeflow Pipelines deployment requires more than 4 GB of RAM, and 30 GB of disk space, so if you are using Docker Desktop make sure to update the resource limits in the preferences.

When using the Kubeflow orchestrator with a remote cluster, you'll additionally need:

A remote ZenML server deployed to the cloud. See the deployment guide for more information.

Kubeflow pipelines deployed on a remote cluster. See the deployment section for more information.