Thread
#prefect-community
    m

    Mitchell Bregman

    2 years ago
    Hi there, has anyone done a performance analysis comparing Prefect to Airflow? Is there some sort of white paper I can use to present to my team?
    james.lamb

    james.lamb

    2 years ago
    m

    Mitchell Bregman

    2 years ago
    Hey James! Yes, this is the guiding document to convincing my team that Prefect > Airflow 😆 , but more so looking for a speed analysis of similar flows.
    Do you know if anyone has performed such an analysis?
    Chris White

    Chris White

    2 years ago
    I’m not aware of anyone who has done a proper speed comparison, but that is most likely because the architecture differences between Airflow and Prefect are so large that it doesn’t seem like a dimension that needs to be tested empirically.  Airflow tasks take a minimum of 10 seconds to be scheduled and run, so two sequential tasks will take a minimum of 20 seconds just to be queued for execution.  Prefect, on the other hand, does not bog the scheduler down with these details and can consequently execute sequential tasks in ~100 milliseconds.  Similar comparisons can be made for parallel pipelines.
    m

    Mitchell Bregman

    2 years ago
    @Chris White copy and pasting this into my PPT deck 😉 Thank you so much for all you do!!! I ❤️ Prefect
    Chris White

    Chris White

    2 years ago
    no problem! Let us know how we can help 😄
    m

    Mitchell Bregman

    2 years ago
    @curtis smiley ^^