https://prefect.io logo
y

Yuliy Kuroki

04/14/2021, 1:54 PM
Hello everyone! Is there a way to have a prefect local agent wait for all tasks it’s running to finish before starting to work on a new task? It’s running a very slow and intensive process and I want it to finish before starting work on a new one.
k

Kevin Kho

04/14/2021, 2:09 PM
Hi @Yuliy Kuroki! I think the only suggestion I have right now is Task concurrency limiting. This is only available in the paid tiers of cloud though. Otherwise, I think what you have to do is just set them as dependencies of each other so they run sequentially.
y

Yuliy Kuroki

04/14/2021, 2:15 PM
Can I get the currently running flows dynamically at launch and set it as a dependency of the flow?
k

Kevin Kho

04/14/2021, 2:19 PM
Not quite but you can make a new Flow that just calls other flows with
StartFlowRun
as a task and then chain them together as dependencies. If you want something more dynamic, I would say use the
Client
to query for flow run state. You may want the
prefect.client.client.Client.get_flow_run_state
. And then you can start the more resource intensive tasks if the state is SUCCESS?
y

Yuliy Kuroki

04/14/2021, 6:19 PM
Thanks! I’ll try that