Hey guys, I'm a long time airflow user and considering prefect. How should I think about prefect? what are the main use cases I should have in mind when choosing Prefect over airflow?

I don't have any links to case studies on hand, but the biggest thing in terms of outcomes is speed of development and increasing the number of people who are able to contribute. Prefect 2 is essentially straight python with very few framework-specific constructs when writing the actual pipelines. If you can reason in python and use python's control flow constructs (loops, if/else, etc.), you can write flows in Prefect 2. Airflow requires knowing both airflow and whatever constructs your org has built around airflow to make it handle production use cases. That means it takes a while to get people up to speed. Or you have to have somebody who knows both airflow and the supplied pipeline code well enough to translate it to airflow.

Hi Adrian, Airflow has improved their syntax a bit and now it is pretty similar to Prefect with decorator-like syntax. The similarity in how work-flows are defined is deceptive though. Although an Airflow script and a Prefect script look pretty similar, with Airflow you’re still unable to run the flows locally by just calling the python function. In prefect you can call the python function to run flows and tasks locally, no boiler-plate code required. This feature of Prefect 2 enables a few key user stories: • You can test your flows and tasks in the same way that you test python functions. • You can step through flows and tasks like you would step through python functions — so using your fave debugger is possible. • Beyond dev and testing, this feature enables you to execute flows in a serverless fashion without the requirement of a Prefect agent.

what are the main use cases I should have in mind when choosing Prefect over airflow?

I'm not sure whether you are asking yourself the right questions. what problem are you trying to solve? what are you trying to accomplish?