on file_that_runs_a_zenml_pipeline.py

Kubeflow UIKubeflow comes with its own UI that you can use to find further details about your pipeline runs, such as the logs of your steps. For any runs executed on Kubeflow, you can get the URL to the Kubeflow UI in Python using the following code snippet:

from zenml.client import Client

pipeline_run = Client().get_pipeline_run("<PIPELINE_RUN_NAME>")

orchestrator_url = pipeline_run.run_metadata["orchestrator_url"].value

Additional configuration

For additional configuration of the Kubeflow orchestrator, you can pass KubeflowOrchestratorSettings which allows you to configure (among others) the following attributes:

client_args: Arguments to pass when initializing the KFP client.

user_namespace: The user namespace to use when creating experiments and runs.

pod_settings: Node selectors, affinity, and tolerations to apply to the Kubernetes Pods running your pipeline. These can be either specified using the Kubernetes model objects or as dictionaries.

from zenml.integrations.kubeflow.flavors.kubeflow_orchestrator_flavor import KubeflowOrchestratorSettings

from kubernetes.client.models import V1Toleration

kubeflow_settings = KubeflowOrchestratorSettings(

client_args={},

user_namespace="my_namespace",

pod_settings={

"affinity": {

"nodeAffinity": {

"requiredDuringSchedulingIgnoredDuringExecution": {

"nodeSelectorTerms": [

"matchExpressions": [

"key": "node.kubernetes.io/name",

"operator": "In",

"values": ["my_powerful_node_group"],

},

"tolerations": [

V1Toleration(

key="node.kubernetes.io/name",

operator="Equal",

value="",

effect="NoSchedule"

@pipeline(

settings={

"orchestrator.kubeflow": kubeflow_settings

...

Check out the SDK docs for a full list of available attributes and this docs page for more information on how to specify settings.

Enabling CUDA for GPU-backed hardware