dammy arinde
01/03/2022, 6:10 PMTypeError: 'Checkpoint' object is not subscriptable
Kevin Kho
dammy arinde
01/03/2022, 6:15 PMKevin Kho
dammy arinde
01/03/2022, 6:17 PMdammy arinde
01/03/2022, 6:17 PMdammy arinde
01/07/2022, 3:07 PMalex
01/07/2022, 3:10 PMgreat_expectations
do you have installed?dammy arinde
01/07/2022, 3:15 PM0.13.47
alex
01/07/2022, 3:17 PMdammy arinde
01/07/2022, 3:17 PMalex
01/07/2022, 3:20 PMdammy arinde
01/07/2022, 3:21 PMdammy arinde
01/07/2022, 3:21 PMTask 'RunGreatExpectationsValidation': Exception encountered during task execution!
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 880, in get_task_run_state
logger=self.logger,
File "/usr/local/lib/python3.7/site-packages/prefect/utilities/executors.py", line 454, in run_task_with_timeout
return task.run(*args, **kwargs) # type: ignore
File "/usr/local/lib/python3.7/site-packages/prefect/utilities/tasks.py", line 456, in method
return run_method(self, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/prefect/tasks/great_expectations/checkpoints.py", line 246, in run
for batch in ge_checkpoint["batches"]:
TypeError: 'Checkpoint' object is not subscriptable
alex
01/07/2022, 3:25 PMdammy arinde
01/07/2022, 3:25 PMKevin Kho
dammy arinde
01/07/2022, 3:41 PMKevin Kho
dammy arinde
01/07/2022, 3:49 PMdammy arinde
01/07/2022, 3:49 PMdammy arinde
01/07/2022, 3:50 PM{
"env": null,
"type": "KubernetesRun",
"image": "molsonlabs/prefect-snowflake:0.0.9",
"labels": [],
"cpu_limit": null,
"__version__": "0.15.11",
"cpu_request": null,
"job_template": null,
"memory_limit": null,
"memory_request": null,
"image_pull_policy": null,
"job_template_path": "<s3://iddoc-shared-dev/prefect/k8s-templates/ge_template.yaml>",
"image_pull_secrets": null,
"service_account_name": null
}
Kevin Kho
dammy arinde
01/07/2022, 4:00 PMKevin Kho
dammy arinde
01/07/2022, 4:11 PMKevin Kho
alex
01/07/2022, 4:41 PMmolsonlabs/prefect-snowflake:0.0.9
is?dammy arinde
01/19/2022, 4:53 PMdammy arinde
01/19/2022, 4:54 PMdammy arinde
01/19/2022, 4:54 PMTask 'RunGreatExpectationsValidation': Exception encountered during task execution!
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 880, in get_task_run_state
logger=self.logger,
File "/usr/local/lib/python3.7/site-packages/prefect/utilities/executors.py", line 467, in run_task_with_timeout
return task.run(*args, **kwargs) # type: ignore
File "/usr/local/lib/python3.7/site-packages/prefect/utilities/tasks.py", line 456, in method
return run_method(self, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/prefect/tasks/great_expectations/checkpoints.py", line 245, in run
run_id={"run_name": run_name or prefect.context.get("task_slug")},
File "/usr/local/lib/python3.7/site-packages/great_expectations/core/usage_statistics/usage_statistics.py", line 307, in usage_statistics_wrapped_method
result = func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/great_expectations/checkpoint/checkpoint.py", line 390, in run
validation_dict=validation_dict,
File "/usr/local/lib/python3.7/site-packages/great_expectations/checkpoint/checkpoint.py", line 240, in _run_validation
else None
File "/usr/local/lib/python3.7/site-packages/great_expectations/data_context/data_context.py", line 1732, in get_validator
expectation_suite = self.get_expectation_suite(expectation_suite_name)
File "/usr/local/lib/python3.7/site-packages/great_expectations/data_context/data_context.py", line 2204, in get_expectation_suite
"expectation_suite %s not found" % expectation_suite_name
great_expectations.exceptions.exceptions.DataContextError: expectation_suite cms_ge_exp not found
dammy arinde
01/19/2022, 4:55 PMalex
01/19/2022, 5:09 PMgreat_expections
folder from S3 in a earlier task in your flow?dammy arinde
01/19/2022, 5:17 PMdammy arinde
01/19/2022, 5:19 PMs3FileName = 'cms2_ge_exp.json'
s3CheckPoint = 'cms_ge_checkpoint.yml'
fileReadPath = WORKING_DIR + '/'+ s3FileName
CPReadPath = WORKING_DIR + '/checkpoints/' + s3CheckPoint
<http://logger.info|logger.info>('fileReadPath: ' + fileReadPath)
s3CopyData = s3Copy('iddoc-shared-dev','prefect/ge/' + s3FileName, fileReadPath)
s3CopyDataCP = s3Copy('iddoc-shared-dev','prefect/ge/' + s3CheckPoint, CPReadPath)
alex
01/20/2022, 2:39 PMexpectations
nested under WORKING_DIR
? Also, your expectation suite file is named cms2_ge_exp.json
, but the error that you saw looks like you’ve configured the expectation suite name to be cms_ge_exp
. Those names should match so that GE can correctly find the expectation suite.dammy arinde
01/20/2022, 2:54 PMdammy arinde
01/20/2022, 2:54 PMfileReadPath = WORKING_DIR + '/expectations/' + s3FileName
dammy arinde
01/20/2022, 2:54 PMdammy arinde
01/20/2022, 2:56 PMdammy arinde
01/20/2022, 2:58 PMdammy arinde
01/20/2022, 2:59 PMdammy arinde
01/20/2022, 3:09 PMdammy arinde
01/20/2022, 3:09 PM