Hey there 🙂
Is there any way in Prefect 1.0 to prioritize task for the execution?
I have a flow with 1k tasks and some dependencies. Most of time hundreds of task are ready for execution, but i use just 20 workers (DaskExecutor).
These tasks can be grouped and i would like to finish one group before continue with the next group.
This could be easily done, if I would be able to set a priority for each task.
I guess if this is not possible I have to build these groups with tasks for just modelling this groups with dependencies. Like:
• 1: Start group 1
• 2-5: Task a,b,c (waits for 1)
• 6: End group 1 (waits for 2-5)
• 7:Start group 2 (waits for 6)
• 8-10: Task x,y,z (waits for 7)
• 11 End group 2 (waits 8-10)
But this is a little bit ugly.
Thanks for any help!
Sadly we can not migrate to orion so soon... 😞