Hi all, I just got started on prefect (I mostly us...
# ask-community
j
Hi all, I just got started on prefect (I mostly used AF before for orchestration) and it's great ❤️ Two quick questions: • I was able to manually write Python code that runs my great expectations checkpoints - the GE built in task does not work with my LegacyCheckpoint, and I wasn't able to make even a simple flow run (below). Did anybody get the GE integration work? Is that worth the trouble, when full "verbose" GE code to run validation is 4 lines of Python? • I re-build the DBT cloud client from AF to work as a Prefect task (two tasks really, scheduling the job, and polling DB cloud for the JOB status) - as I prefer DB cloud to DB shell locally installed. Is there any interest from the community in this sort of integration? If yes, happy to chat / share the simple POC I built. GE CODE THAT DOES NOT WORK
Copy code
from prefect import task, Flow, Parameter
from prefect.tasks.great_expectations import RunGreatExpectationsValidation

# Define checkpoint task
validation_task = RunGreatExpectationsValidation()

with Flow("ge_test") as flow:
    validation_task(checkpoint_name='gitter_checkpoint')

flow.run()
👀 2
n
Hi @Jacopo Tagliabue - can you post the error you're seeing with your GE task? An alternative to the GE task would be to pipe markdown output from your custom task directly to the
artifacts.create_markdown
function to generate the UI artifact
j
this is the error, and that is the line in the
debug mode
the ge checkpoint in the debugger cannot access the
batches
by string indexing
n
Hm got it - I'm not super familiar with GE but if you've got your own GE code working, great! Are you also trying to generate an artifact for your validation?
j
artifact
= HTML viz? No, but would be lovely to have it in the cloud (I mean, I created it, but locally, NOT in prefect cloud)