Hi,
Is there an equivalent of Airflow’s `TaskGroup`’s in Prefect? Or is there any other way to build larger flows (100+ tasks) while making it easy to navigate the flow in the UI?
I think I’ve seen something similar before, but I can’t find any references…
Thanks
k
Kevin Kho
10/22/2021, 6:02 PM
Hey @Erik Nilsson, so the quick answer is there isn’t really anything that will help navigating in the UI. You could break it down into subflows and have a Flow of Flows setup, but the task graph will then be cut up and you’ll see them in separate flow runs. We have gotten a lot of feedback around this so in Prefect 2.0, there will be a new schematic that scales better and subflows will also be connected to the main flow.
e
Erik Nilsson
10/22/2021, 6:22 PM
Thanks for the quick response. Looking forward to 2.0. I’ll checkout flow of flows in the meanwhile.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.