e going to be starting with a very basic approach.First, let's understand what a service connector does. In simple words, a service connector contains credentials that grant stack components access to cloud infrastructure. These credentials are stored in the form of a secret, and are available to the ZenML server to use. Using these credentials, the service connector brokers a short-lived token and grants temporary permissions to the stack component to access that infrastructure. This diagram represents this process:

There are many ways to create an AWS service connector, but for the sake of this guide, we recommend creating one by using the IAM method.

AWS_PROFILE=<AWS_PROFILE> zenml service-connector register cloud_connector --type aws --auto-configure

There are many ways to create a GCP service connector, but for the sake of this guide, we recommend creating one by using the Service Account method.

zenml service-connector register cloud_connector --type gcp --auth-method service-account --service_account_json=@<PATH_TO_SERVICE_ACCOUNT_JSON> --project_id=<PROJECT_ID> --generate_temporary_tokens=False

There are many ways to create an Azure service connector, but for the sake of this guide, we recommend creating one by using the Service Principal method.

zenml service-connector register cloud_connector --type azure --auth-method service-principal --tenant_id=<TENANT_ID> --client_id=<CLIENT_ID> --client_secret=<CLIENT_SECRET>

Once we have our service connector, we can now attach it to stack components. In this case, we are going to connect it to our remote artifact store:

zenml artifact-store connect cloud_artifact_store --connector cloud_connector

Now, every time you (or anyone else with access) uses the cloud_artifact_store, they will be granted a temporary token that will grant them access to the remote storage. Therefore, your colleagues don't need to worry about setting up credentials and installing clients locally!

Running a pipeline on a cloud stack