Hi everyone! I’m trying to better understand the Great Expectations plugin for Prefect. I currently have a ETL pipeline I’d like to Prefect-ify into tasks, which doesn’t sound too difficult. I would also want to use Great Expectations to declare expectations around any given task’s inputs and outputs (e.g., assert some input’s column is unique, or that the output table doesn’t drop any rows). My question is: is this when I’d use the Great Expectations plugin? As far as I can tell, I don’t need the heavier offerings from Great Expectations; just inline data assertions to make sure my pipeline is “correct”. Please let me know if this question is better suited for a different channel or the Great Expectations Slack group. Thank you!

Hey Danny! This like a reasonable place to start using GE. Laying the groundwork now will likely pay off later when you do reach for some of the heavier features 👍

@Dylan Totally agree! I guess my question was: do I need the GE plugin for my (relatively simple) use-case? Would you happen to know if there’s a GE plugin (presumably for Prefect’s class-based tasks?) that can help me do this? I’m happy to write one on my own, if it doesn’t exist already 🙂