c

    Christian

    5 months ago
    Hi all 👋 Just wondering: are 3rd party tasks (i.e. the Great Expectations validation) compatible with Prefect Orion? Anyone using this? I’d want to validate a dataframe from a upstream task with GE in a flow… Cheers, C
    Anna Geller

    Anna Geller

    5 months ago
    They are not, but Prefect 2.0 has something way cooler than just a task library - we now have Prefect Collections! They are more powerful since: • they live in separate repos making them more composable and modular with your custom Prefect setup (also preventing dependency conflicts) • they contain more than just custom tasks GE is on the roadmap - check out available collections here
    c

    Christian

    5 months ago
    Thanks for the info. Any hack/ bridge to get things running in the meantime. Or are those things plain incompatible?
    Anna Geller

    Anna Geller

    5 months ago
    You are welcome to help contribute to the GE collection if you want to. Or you can check the code in 1.0 and port it over to your custom 2.0 task. Afaik GE validation could be done with a subprocess calling the GE CLI command cc @alex
    c

    Christian

    5 months ago
    Thanks. I’ll take a look