https://prefect.io logo
Title
t

Tony Alfonse

01/26/2023, 7:09 PM
Hey Community! Is there a way to wait for another deployment's flow to finish before doing something in our main flow? i.e. I have a deployment/pipeline with three sub-flows A, B, and C, I want to run sub-flow A first, then it should check if another deployment's main flow has finished or not if it has it should trigger sub-flow B. More like a wait_for functionality but instead of tasks or flows within the same deployment, it should wait for another deployment's main flow to finish first. I am able to come up with a solution using Prefect Orion API where my deployment A takes the name of the second deployment's flow on which sub-flow B depends and then it checks the last execution of the second deployment's flow's state for the current day, if the state is finished, it would go on to execute the sub-flow B, if it's in any of the pending states, it would just hold on and retry for an hour and wait until it completes. it's working okay for now but I think this is a naive approach. I was just wondering if there is any prefect native approach to do this?
n

Nate

01/26/2023, 7:15 PM
Hi @Tony Alfonse by default, the
run_deployment
utility will wait for the completion of the flow run that it creates (you can pass timeout = 0 to fire and forget) here's a discussion in an older thread you may find useful - let me know if you'd like details on what I outlined
t

Tony Alfonse

01/26/2023, 7:24 PM
Thanks @Nate, let me check.
Hey @Nate, I checked the discussion you referenced and the example assumes you can run the second deployment/flow, however, in my case I have no control over the second deployment's execution and it would be triggered from another upstream process, I just have to check its state and wait for it to finish if it hasn't. Any leads?
n

Nate

01/26/2023, 8:35 PM
ahh I see does subflow A need to run any particular time relative to the second deployment's flow run? I guess I'm wondering if you can just trigger the deployment that runs A > B > C at the end of the second deployment, then you wouldn't have to poll for state of the second deployment between A and B if you have no control over the second deployment's execution, I'd say that some version of your approach is likely what I would settle on as well (based on my understanding of what you laid out)
t

Tony Alfonse

01/26/2023, 8:40 PM
I guess I would've to settle down on this approach then! šŸ™‚ Thank you nonetheless and have a good day!
n

Nate

01/26/2023, 8:47 PM
sure thing! just a couple more thoughts regarding some new work we'll be rolling out relatively soon I think in the quite near future, this could be addressed using automations, which would look like: ā€¢ ABC flow run starts: ā—¦ A finishes, ā—¦ ABC flow run checks once for state of 2nd deployment. continues if 2nd deployment is done, enters PAUSED state if not ā€¢ automation is configured to place ABC into a RUNNING state upon the completion of the 2nd deployment
t

Tony Alfonse

01/26/2023, 9:12 PM
Tbh, that'd be cool!
Hey @Nate, was this feature released in the latest release?
n

Nate

02/08/2023, 1:23 AM
Hey @Tony Alfonse - not yet, sorry. I've raised this internally as something that people would like to see, but the "place existing flow run into X state" action hasn't been prioritized yet let me know if you have any issues with your solution in the meantime
t

Tony Alfonse

02/09/2023, 10:16 PM
Thank you! Looking forward to its implementation. I think my solution is working fine for now.
šŸ‘ 1