Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

I'd be very curious to discuss a few points/user experience with a couple of people if possible/before committing.

Hi <@UPWTZFHM2>, we've been using Prefect Core (~4 months) and Cloud (~2 months) and I'd be happy to answer any questions and/or share our experience. (Our previous experience with data pipelines was using Airflow for about 4 years: we have hundreds of Airflow DAGs, etc.) Let me know what would be most useful for you to know.

Have you completely moved away from Airflow in favour of Prefect now?

<@UPWTZFHM2> we are in the process of adopting Prefect, but for us it'll be a long process just since we have so many Airflow DAGs that we've built over 4 years. Right now we're using Prefect mainly for data science pipelines, while the existing data engineering pipelines remain on Airflow. Long-term though I don't see any reason why we couldn't use Prefect for all of our data pipelines, both for data engineering and data science.