Hey community! I’m trying to set up Prefect with GreatExpectations, using the official task. I’m trying to run a validation on an in-memory Dataframe (result of previous task) but have trouble to set it up correctly. I’m trying to use v3 API of GE, set up the expectation suite, checkpoint according to the guide (until _context.run_checkpoint)_ but struggle to pass said dataframe. Would anyone be able to offer some guidance?

I don’t think you pass an in memory dataframe for that task. You point to a DataFrame in storage and run the checkpoint against it

Thanks for getting back Kevin! What do you mean by DataFrame in storage? A saved csv/parquet file?

Yeah exactly

Alright, that makes things much easier, I guess I was trying to complicate my life for no big reason. Thanks Kevin 👌 , will run with that and let know how it went.

Nice! Thanks for circling back