g_suite: bool = True,

) -> ExpectationSuite:

...You can view the complete list of configuration parameters in the SDK docs.

The Great Expectations data validator step

The standard Great Expectations data validator step validates an input pandas.DataFrame dataset by running an existing Expectation Suite on it. The validation results are saved in the Great Expectations Validation Store, but also returned as an CheckpointResult artifact that is versioned and saved in the ZenML Artifact Store. The step automatically rebuilds the Data Docs.

At a minimum, the step configuration expects the name of the Expectation Suite to be used for the validation:

from zenml.integrations.great_expectations.steps import (

great_expectations_validator_step,

ge_validator_step = great_expectations_validator_step.with_options(

parameters={

"expectation_suite_name": "steel_plates_suite",

"data_asset_name": "steel_plates_train_df",

The step can then be inserted into your pipeline where it can take in a pandas dataframe and a bool flag used solely for order reinforcement purposes, e.g.:

docker_settings = DockerSettings(required_integrations=[SKLEARN, GREAT_EXPECTATIONS])

@pipeline(settings={"docker": docker_settings})

def validation_pipeline():

"""Data validation pipeline for Great Expectations.

The pipeline imports a test data from a source, then uses the builtin

Great Expectations data validation step to validate the dataset against

the expectation suite generated in the profiling pipeline.

Args:

importer: test data importer step

validator: dataset validation step

checker: checks the validation results

"""

dataset, condition = importer()

results = ge_validator_step(dataset, condition)

message = checker(results)

validation_pipeline()