https://prefect.io logo
Title
m

Marcus Hughes

08/27/2021, 3:42 PM
I see in Dask that there's a priority parameter for "when there's more work than workers [and] Dask has to decide which tasks to prioritize over others." Is this exposed to Prefect? More specifically, let's say I have a collection of flows and my system is running low on resources, is there a way to tell Prefect to prioritize completing one flow over another? Thanks in advance!
n

nicholas

08/27/2021, 3:44 PM
Hi @Marcus Hughes - this isn't possible in Prefect at the moment but if it's something you'd like to see it's worth opening a discussion in Github 🙂
k

Kevin Kho

08/27/2021, 4:41 PM
So there is this issue for task priority you can chime in. For the Flow level though, I think this is tricky because having a priority is a task queue kind of thing where you have a set of tasks that feed into a set of workers, and then you can set a “priority queue” On the flow level, Prefect isn’t meant to be a task queue (or flow queue) so it will just attempt to start whatever. I know it’s not quite what you want, but have you seen flow concurrency limits?
:upvote: 1