Hey all, I'm Gareth - Founder of Ritza.co, a technical publishing company. We produce books, blog posts, technical documentation etc with a focus on machine learning, devops and software engineering.
I'm writing some articles that discuss Prefect so would love to discuss some things here as I don't have a ton of first hand experience with it. Specifically regarding the "why not airflow" article and whether Airflow 2.0 adequately addresses any of the concerns about Airflow raised there.
https://docs.prefect.io/core/getting_started/why-not-airflow.html
Specifically I'd like to add prefect to our article here https://www.datarevenue.com/en-blog/airflow-vs-luigi-vs-argo-vs-mlflow-vs-kubeflow
đ 1
đ 3
d
Dylan
12/30/2020, 4:45 PM
Hey @Gareth Dwyer, welcome to the community!
This is the place to ask questions and learn more about Prefect! Let us know how we can help đ
c
Chris White
12/30/2020, 6:05 PM
Hi @Gareth Dwyer! The short and sweet is that while Airflow 2.0 did introduce some performance improvements and nice API enhancements (which may look surprisingly familiar to Prefect users haha), the thrust of the original âWhy Not Airflowâ post was about core architectural constraints, none of which were addressed or modified for 2.0. As such, we donât see any new use cases that Airflow 2.0 can handle that Airflow 1.0 couldnât, but Iâm sure the enhancements were nice for already existing Airflow users. (And to be honest, I get the sense that â2.0" was used more for marketing purposes than to signify features worthy of such a version bump).
Iâll DM you with my contact info if youâd like to dig deeper / go point-by-point or anything - Iâm definitely happy to help you in whatever way makes sense!
g
Gareth Dwyer
12/31/2020, 7:14 AM
@Chris White - amazing - exactly the info I was looking for đ - would love to chat more too. Thank you!