ons configuration is a file on your local machine.migrate your existing Great Expectations configuration to ZenML. This is a compromise between 1. and 2. that allows you to continue to use your existing Data Sources, Metadata Stores and Data Docs sites even when running pipelines remotely.

Some Great Expectations CLI commands will not work well with the deployment methods that puts ZenML in charge of your Great Expectations configuration (i.e. 1. and 3.). You will be required to use Python code to manage your Expectations and you will have to edit the Jupyter notebooks generated by the Great Expectations CLI to connect them to your ZenML managed configuration. .

The default Data Validator setup plugs Great Expectations directly into the Artifact Store component that is part of the same stack. As a result, the Expectation Suites, Validation Results and Data Docs are stored in the ZenML Artifact Store and you don't have to configure Great Expectations at all, ZenML takes care of that for you:

# Register the Great Expectations data validator

zenml data-validator register ge_data_validator --flavor=great_expectations

# Register and set a stack with the new data validator

zenml stack register custom_stack -dv ge_data_validator ... --set

If you have an existing Great Expectations configuration that you would like to reuse with your ZenML pipelines, the Data Validator allows you to do so. All you need is to point it to the folder where your local great_expectations.yaml configuration file is located:

# Register the Great Expectations data validator

zenml data-validator register ge_data_validator --flavor=great_expectations \

--context_root_dir=/path/to/my/great_expectations

# Register and set a stack with the new data validator

zenml stack register custom_stack -dv ge_data_validator ... --set