Hi all, I have a question regarding using great e...
# prefect-community
y
Hi all, I have a question regarding using great expectations with prefect in production environment, I have an agent running on EC2 and flow code stored in S3. However, whenever I deploy and run a flow that has great_expectation's
run_checkpoint_validation
I get the following error but it runs fine locally. Also, the s3 storage used to store flow code has the great_expectations directory in it..
a
Hey @Yousef Hosny! What type of datasource is
landing_zone
?
y
Hi Alex, It's Athena
Also, to give more context; great_expectations is installed in the prefect agent ec2 instance, I don't know what might be the issue..
a
Could there be a difference in the version of
great_expectations
installed locally vs. the version installed on your EC2 instance?
y
Same version, 0.15.20
a
Hmmm, the next thing I would recommend checking would be if you can run your great expectations validation on that EC2 instance via
great_expectations
directly.
j
Not sure, looks like a path reference problem. Maybe try to pass the path to the dir as an argument (not sure which package are you using, but something like
data_context_root_dir
)
y
I did that, I even created a data context with the path and passed it to
data_context
, still no success..
I have to understand the mechanism of prefect agent accessing the S3 flow code storage block, is there any reference to this, I tried searching the docs but couldn't find any
t
I'm having the same issue. Was this resolved?