https://prefect.io logo
Title
j

Jason Motley

03/22/2022, 2:30 PM
I know that you can set retries on individual task failures. Can you set a retry on a "flow of flows" if one flow fails?
k

Kevin Kho

03/22/2022, 2:33 PM
It’s hard because retrying a
create_flow_run
with the same idempotency key just returns the result of what was already run. So you need to restart the Flow in the UI on the subflow, and then restart the main Flow. We don’t have restart through the API, only the UI has it. What you can do though is create a new flow run with a new idempotency key and then rely on caching so that you don’t repeat work that is already done
j

Jason Motley

03/22/2022, 2:36 PM
Is there an example of how that would work?
k

Kevin Kho

03/22/2022, 2:39 PM
Man it’s been a while I don’t think so but there are three parts. First is the idempotency key, you want to fill it with a different value to trigger another flow run
with Flow(..) as flow:
    create_flow_run(..., idempotency_key=datetime.datetime.now())
Second is that you actually want this value to change from run to run instead of being fixed at build time so you need script-based storage (Github, etc). and then on the subflow side, apply
cache_for
and
cache_validator
to the expensive tasks so they don’t rerun
@task(cache_for=...)
def myfunc(x):
    return x + 1
All this will create a new flow run when you restart but respect the cached work. Or you can click Restart in the UI for the subflow, then click it for the main flow and that should work
j

Jason Motley

03/22/2022, 2:41 PM
Okay this is very helpful! Would we have to manually restart the flow run?
Or does the indempendency do that?
k

Kevin Kho

03/22/2022, 2:42 PM
Still yes in both cases. idempotency just helps create a new child flow run because using the same idempotency key does nothing
And this is all much improved in Prefect 2.0 one way or another, especially retrying subflows
j

Jason Motley

03/22/2022, 2:43 PM
Gotcha, okay this is great man thank you!~
k

Kevin Kho

03/22/2022, 2:43 PM
How many subflows does your main flow have?
j

Jason Motley

03/22/2022, 2:44 PM
running 4 flows via orchestration
k

Kevin Kho

03/22/2022, 2:45 PM
ok yeah that’s not too bad. i know a guy who had 20
and i know this is painful
j

Jason Motley

03/22/2022, 2:45 PM
haha well maybe waiting for prefect 2.0 is the move