│ s3://zenml-public-datasets                     ┃┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

After having set up or decided on an AWS Service Connector to use to connect to the target S3 bucket, you can register the S3 Artifact Store as follows:

# Register the S3 artifact-store and reference the target S3 bucket

zenml artifact-store register <S3_STORE_NAME> -f s3 \

--path='s3://your-bucket'

# Connect the S3 artifact-store to the target bucket via an AWS Service Connector

zenml artifact-store connect <S3_STORE_NAME> -i

A non-interactive version that connects the S3 Artifact Store to a target S3 bucket through an AWS Service Connector:

zenml artifact-store connect <S3_STORE_NAME> --connector <CONNECTOR_ID>

Example Command Output

$ zenml artifact-store connect s3-zenfiles --connector s3-zenfiles

Successfully connected artifact store `s3-zenfiles` to the following resources:

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┓

┃             CONNECTOR ID             │ CONNECTOR NAME │ CONNECTOR TYPE │ RESOURCE TYPE │ RESOURCE NAMES ┃

┠──────────────────────────────────────┼────────────────┼────────────────┼───────────────┼────────────────┨

┃ c4ee3f0a-bc69-4c79-9a74-297b2dd47d50 │ s3-zenfiles    │ 🔶 aws         │ 📦 s3-bucket  │ s3://zenfiles  ┃

┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛

As a final step, you can use the S3 Artifact Store in a ZenML Stack:

# Register and set a stack with the new artifact store

zenml stack register <STACK_NAME> -a <S3_STORE_NAME> ... --set

When you register the S3 Artifact Store, you can generate an AWS access key, store it in a ZenML Secret and then reference it in the Artifact Store configuration.

This method has some advantages over the implicit authentication method:

you don't need to install and configure the AWS CLI on your host