that might be useful for your particular use-case:configure_zenml_stores: if set, ZenML will automatically update the Great Expectation configuration to include Metadata Stores that use the Artifact Store as a backend. If neither context_root_dir nor context_config are set, this is the default behavior. You can set this flag to use the ZenML Artifact Store as a backend for Great Expectations with any of the deployment methods described above. Note that ZenML will not copy the information in your existing Great Expectations stores (e.g. Expectation Suites, Validation Results) in the ZenML Artifact Store. This is something that you will have to do yourself.

configure_local_docs: set this flag to configure a local Data Docs site where Great Expectations docs are generated and can be visualized locally. Use this in case you don't already have a local Data Docs site in your existing Great Expectations configuration.

For more, up-to-date information on the Great Expectations Data Validator configuration, you can have a look at the SDK docs .

How do you use it?

The core Great Expectations concepts that you should be aware of when using it within ZenML pipelines are Expectations / Expectation Suites, Validations and Data Docs.

ZenML wraps the Great Expectations' functionality in the form of two standard steps:

a Great Expectations data profiler that can be used to automatically generate Expectation Suites from an input pandas.DataFrame dataset

a Great Expectations data validator that uses an existing Expectation Suite to validate an input pandas.DataFrame dataset

You can visualize Great Expectations Suites and Results in Jupyter notebooks or view them directly in the ZenML dashboard.

The Great Expectation's data profiler step