👋 hello all! I’ve been playing around with the great expectations validation task using the documentation here: https://docs.prefect.io/api/latest/tasks/great_expectations.html#rungreatexpectationsvalidation. I don’t have

validation_operators

in my

great_expectations.yml

file so I call the task as follows:

validation_task = RunGreatExpectationsValidation()
    validation_results = validation_task(
        batch_kwargs=validation.get_batch_kwargs(records, 'pandas_s3'), expectation_suite_name=GE_EXPECTATION_SUITE_NAME, validation_operator=None
    )

However when I execute my flow I get the following error:

Unexpected error: DataContextError('No validation operator `None` was found in your project. Please verify this in your great_expectations.yml')
Traceback (most recent call last):
  File "/Users/soren.daugaard/Code/data-pipelines/.direnv/python-3.8.6/lib/python3.8/site-packages/great_expectations/data_context/data_context.py", line 1272, in run_validation_operator
    validation_operator = self.validation_operators[validation_operator_name]
KeyError: None

According to the documentation that parameter should be optional, but I am not able to figure out how to set it to nothing. I tried

None

and the empty string but with similar errors. Any pointers would be much appreciated.

Hey @Soren Daugaard, Is using the default

action_list_operator

an option for you? This task may be configured to allow this default to be overridden by your yaml, even if a validation operator is not used. (*I am not a GE’er…)

Ah forgot to mention the default would throw an error also like this:

[2021-03-08 16:52:48-0500] ERROR - prefect.TaskRunner | Unexpected error: DataContextError('No validation operator `action_list_operator` was found in your project. Please verify this in your great_expectations.yml')
Traceback (most recent call last):
  File "/Users/soren.daugaard/Code/data-pipelines/.direnv/python-3.8.6/lib/python3.8/site-packages/great_expectations/data_context/data_context.py", line 1272, in run_validation_operator
    validation_operator = self.validation_operators[validation_operator_name]
KeyError: 'action_list_operator'

Hmm, unfortunately I’m not seasoned enough with GE to provide proper guidance here, but I will continue digging.

Ah good question @Kyle Moon-Wright. We’re still using GE for validation. The validation operators are a way to define custom logic around the validation. We don’t use validation operators because they are being deprecated in the GE are being replaced by checkpoints (see https://docs.greatexpectations.io/en/0.13.8/reference/core_concepts/checkpoints_and_actions.html#checkpoints-and-actions).

Hey Soren - I don’t think there was a specified version for the GE Tasks in the Task Library because this was a user contributed component. From our side, these tasks were introduced in Prefect version 0.13.19 around December 8th 2020 alongside this article, so I’m not sure it’s maintained with every new version of GE.