Hi all, We are looking into Prefect for a data science project. Couple of things are not immediately clear (we have history with Airflow): 1. Is there a concept of a dags/flows folder where Prefect server looks for flow definitions? 2. Is there a possibility on the core UI to authenticate against LDAP?

Hi @Daniel Sali, Prefect’s model is different from Airflow’s in how flows are handled. Airflow needs a DAG folder because every single time your interact with a DAG its code gets parsed and executed. Prefect works differently - you

register

your flow with the server, which sends all the flow metadata and makes it appear in the UI and become available for action. You use Prefect Core’s

Storage

classes to let the server know where the flows live - for example, you could store it locally in any folder you want, or in Docker, or cloud storage, etc. When it’s time to run the flow, the server sends the instruction to your agent, which loads the flow from storage and executes it. The advantage of this model is you never risk odd side effects from re-executing the DAG file. In addition, there are some nice things we can do when you register the flow - for example, you can version your flows to automatically archive an old version and replace it with a new version. Here is the server deployment tutorial, which will walk you through getting started with registering flows against the server / Cloud. Let us know if that’s helpful!

Thanks, @Jeremiah for the explanation, I need to adjust myself a bit 🙂 Auth is not critical to us, Prefect would be internal, just a safety net.

great explanation @Jeremiah! thanks. i too am coming from an airflow world and was wondering how this would work. the prefect UI going open source has gotten me completely on the train

@Daniel Sali what cloud platform are you on?

@Jeremiah Great explanation! It would be nice if the explanation put on Prefect's docs. I think many Prefect users come from Airflow world and stuck by this concept. Users with Airflow experience (including me) tend to lookup docs to let server to parse the definition files of flows, just like Airflow does. But in Prefect the concept is totally different and it's hard for the users to solve this problem from docs without knowing this concept.