rator can be deployed directly from the ZenML CLI:zenml orchestrator deploy sagemaker_orchestrator --flavor=sagemaker --provider=aws ...

You can pass other configurations specific to the stack components as key-value arguments. If you don't provide a name, a random one is generated for you. For more information about how to work use the CLI for this, please refer to the dedicated documentation section.

How to use it

To use the Sagemaker orchestrator, we need:

The ZenML aws and s3 integrations installed. If you haven't done so, run

zenml integration install aws s3

Docker installed and running.

A remote artifact store as part of your stack (configured with an authentication_secret attribute).

A remote container registry as part of your stack.

An IAM role or user with an AmazonSageMakerFullAccess managed policy applied to it as well as sagemaker.amazonaws.com added as a Principal Service. Full details on these permissions can be found here or use the ZenML recipe (when available) which will set up the necessary permissions for you.

The local client (whoever is running the pipeline) will also have to have the necessary permissions or roles to be able to launch Sagemaker jobs. (This would be covered by the AmazonSageMakerFullAccess policy suggested above.)

There are three ways you can authenticate your orchestrator and link it to the IAM role you have created:

The recommended way to authenticate your SageMaker orchestrator is by registering an AWS Service Connector and connecting it to your SageMaker orchestrator:

zenml service-connector register <CONNECTOR_NAME> --type aws -i

zenml orchestrator register <ORCHESTRATOR_NAME> \

--flavor=sagemaker \

--execution_role=<YOUR_IAM_ROLE_ARN>

zenml orchestrator connect <ORCHESTRATOR_NAME> --connector <CONNECTOR_NAME>

zenml stack register <STACK_NAME> -o <ORCHESTRATOR_NAME> ... --set

Instead of creating a service connector, you can also configure your AWS authentication credentials directly in the orchestrator: