I'd be very curious to discuss a few points/user experience with a couple of people if possible/before committing.
10/29/2019, 7:35 PM
Hi @Arlo Bryer, we've been using Prefect Core (~4 months) and Cloud (~2 months) and I'd be happy to answer any questions and/or share our experience. (Our previous experience with data pipelines was using Airflow for about 4 years: we have hundreds of Airflow DAGs, etc.) Let me know what would be most useful for you to know.
10/30/2019, 9:50 AM
Hi @Joe Schmid - great, thanks!
Have you completely moved away from Airflow in favour of Prefect now?
10/30/2019, 2:51 PM
@Arlo Bryer we are in the process of adopting Prefect, but for us it'll be a long process just since we have so many Airflow DAGs that we've built over 4 years. Right now we're using Prefect mainly for data science pipelines, while the existing data engineering pipelines remain on Airflow. Long-term though I don't see any reason why we couldn't use Prefect for all of our data pipelines, both for data engineering and data science.