Amit Gal
05/31/2021, 10:43 AMflow_a = StartFlowRun(flow_name="A", project_name="examples", wait=True)
flow_b = StartFlowRun(flow_name="B", project_name="examples", wait=True)
## Flow of flows
with Flow("parent-flow") as flow:
flow_b.set_upstream(flow_a)
flow.run()
as far as I can see, running this script waits with the scheduling of flow_b
, until flow_a
is completed. Is there a way to schedule them both, but wait with running flow_b
until flow_a
is completed?
My use case is I use a VM to run the flows, but my local machine (or phone) to run a small script that schedules them ad-hoc. flow_a
is often long, while flow_b
is a short one - and so as it currently stands any issues with my local machine might interrupt scheduling flow_b
.
There are a couple of ways I can imagine to solve this, but I wanted to make sure if my understanding is correct, or if there is a way to schedule both at and create a dependency for the running
of the flow?Kevin Kho
flow_b
is scheduled after flow_a
. You want flow_b
to be scheduled so that if flow_a
fails, flow_b
would still trigger right?Kevin Kho
flow_a
to schedule flow_b
whether it fails or succeeds. This won't work though if the local machine just dies completely.Amit Gal
05/31/2021, 4:40 PMflow_b
to run whether flow_a
was successful or not.
However, it should only run after flow_b
finished (successfully or not).
My issue is literally that the scheduling of flow_b
occurs after flow_a
finishes running, while what I’d like to happen is to schedule both of them at the same time, with the logic of “`flow_b` after `flow_a`”, and then I could shut down my local machine.Kevin Kho
wait=False
in the StartFlowRun?Amit Gal
05/31/2021, 5:24 PMwait=False
for flow_b
does schedule both together, but without the logic ensuring flow_b
runs only after flow_a
is finished.
The state_handler
idea is interesting, I am going to look into it and see if I can make it work. Thanks for your thoughts!