a

    Arlo Bryer

    2 years ago
    I'd be very curious to discuss a few points/user experience with a couple of people if possible/before committing.
    j

    Joe Schmid

    2 years ago
    Hi @Arlo Bryer, we've been using Prefect Core (4 months) and Cloud (2 months) and I'd be happy to answer any questions and/or share our experience. (Our previous experience with data pipelines was using Airflow for about 4 years: we have hundreds of Airflow DAGs, etc.) Let me know what would be most useful for you to know.
    a

    Arlo Bryer

    2 years ago
    Hi @Joe Schmid - great, thanks!
    Have you completely moved away from Airflow in favour of Prefect now?
    j

    Joe Schmid

    2 years ago
    @Arlo Bryer we are in the process of adopting Prefect, but for us it'll be a long process just since we have so many Airflow DAGs that we've built over 4 years. Right now we're using Prefect mainly for data science pipelines, while the existing data engineering pipelines remain on Airflow. Long-term though I don't see any reason why we couldn't use Prefect for all of our data pipelines, both for data engineering and data science.