https://prefect.io logo
Title
m

Michael Hadorn

03/10/2023, 2:55 PM
Hey there 🙂 Is there any way in Prefect 1.0 to prioritize task for the execution? I have a flow with 1k tasks and some dependencies. Most of time hundreds of task are ready for execution, but i use just 20 workers (DaskExecutor). These tasks can be grouped and i would like to finish one group before continue with the next group. This could be easily done, if I would be able to set a priority for each task. I guess if this is not possible I have to build these groups with tasks for just modelling this groups with dependencies. Like: • 1: Start group 1 • 2-5: Task a,b,c (waits for 1) • 6: End group 1 (waits for 2-5) • 7:Start group 2 (waits for 6) • 8-10: Task x,y,z (waits for 7) • 11 End group 2 (waits 8-10) But this is a little bit ugly. Thanks for any help! Sadly we can not migrate to orion so soon... 😞
Maybe: is there a channel for prefect 1.0 questions? I could not find it