Hi, looking at Prefect vs Airflow. AWS' MWAA has a mechanism where you can upload a DAG to a designated s3 bucket and it automatically registers with Airflow. I've read a little about the file based flow idioms in prefect, but is there any mechanism to point at say github repo and automatically register the flows? I know it's just one step but if I were to have a lot of flows I'd love to be able to just store my flows in git, point prefect at my repo and have it manage the registrations based on my source
a
Amanda Wee
01/31/2021, 6:27 AM
Perhaps a Github Action could be configured to do the flow registration upon say, a merge to the main branch?
👍 1
j
jspeis
01/31/2021, 2:51 PM
I was thinking about that too
jspeis
01/31/2021, 2:56 PM
would just also want to get a list of all registered flows and and unregister/remove ones no longer in the repo..seems doable with GH Actions, but thought it could be a nice baked in feature too
d
Dylan
02/01/2021, 3:35 PM
Hey Jspeis!
We’re looking into something like this 😉
For now, Amanda’s suggestion is the way to go 👍
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.