https://prefect.io logo
Title
f

Felix Haba

10/12/2022, 5:32 PM
Hi folks - my team is looking into adopting prefect 2.1. One of the functionalities that we need is to be able to run a flow every time some external parameters change (if no flow is being run, run the flow, if a flow is being run, wait until it finishes). I was looking into the concept of work_queues and it satisfies that need mostly - however, we only care about running the latest job in the queue (the one that has the most up to date parameters) is there a way for the agent to always pick the most updated job in the queue and discard the others?
1
k

Kevin Wang

10/12/2022, 6:58 PM
I don't believe there is.. I'm still learning to work with the queues. One way to achieve this is to read your entire queue, then compare them to purge the ones you no longer want executed. https://docs.prefect.io/api-ref/prefect/client/#prefect.client.orion.OrionClient.get_runs_in_work_queue
🙌 1
:thank-you: 1
f

Felix Haba

10/12/2022, 8:19 PM
interesting... and to purge them I would probably do delete_flow_run(id) most likely right? Thanks
k

Kevin Wang

10/12/2022, 8:32 PM
yep, exactly. Another option might be to
set_flow_run_state
to Cancelled. But there is some validation logic in CoreFlowPolicy to make things safe and could prevent this. (I feel like it should be a valid transition to go from scheduled -> cancelled).
f

Felix Haba

10/12/2022, 11:03 PM
got it!