w location. Some examples of such changes include:switching Secrets Store back-end types (e.g. from internal SQL database to AWS Secrets Manager or Azure Key Vault)

switching back-end locations (e.g. changing the AWS Secrets Manager account or region, GCP Secret Manager project or Azure Key Vault's vault).

In such cases, it is not sufficient to simply reconfigure and redeploy the ZenML server with the new Secrets Store configuration. This is because the ZenML server will not automatically migrate existing secrets to the new location. Instead, you should follow a specific migration strategy to ensure that existing secrets are also properly migrated to the new location with minimal, even zero downtime.

The secrets migration process makes use of the fact that a secondary Secrets Store can be configured for the ZenML server for backup purposes. This secondary Secrets Store is used as an intermediate step in the migration process. The migration process is as follows (we'll refer to the Secrets Store that is currently in use as Secrets Store A and the Secrets Store that will be used after the migration as Secrets Store B):

Re-configure the ZenML server to use Secrets Store B as the secondary Secrets Store.

Re-deploy the ZenML server.

Use the zenml secret backup CLI command to back up all secrets from Secrets Store A to Secrets Store B. You don't have to worry about secrets that are created or updated by users during or after this process, as they will be automatically backed up to Secrets Store B. If you also wish to delete secrets from Secrets Store A after they are successfully backed up to Secrets Store B, you should run zenml secret backup --delete-secrets instead.

Re-configure the ZenML server to use Secrets Store B as the primary Secrets Store and remove Secrets Store A as the secondary Secrets Store.

Re-deploy the ZenML server.

This migration strategy is not necessary if the actual location of the secrets values in the Secrets Store back-end does not change. For example: