I have a workflow that writes a file to S3 I then have a flow-of-flows, which runs this workflow via

create_flow_run

, with the

idempotency_key

set to this file path. Running the flow the first time, will write this key. Running it a second time, won't -- as expected, because of the idempotency behavior. What I don't understand is that the second flow run has every task state set to 'success' -- but wouldn't I expect them to be 'cached' or something else, indicating that the flow isn't being re-ran?

You have to re-run on the subflow page, because the

create_flow_run

task is just an API call with the same key that will return the already ran flow

I'm not sure how to get to the subflow page. Clicking on the "task" that amounts to the started flow run, and running 'restart' there, doesn't seem to indicate any cache behavior.

You can navigate to it in the UI. There will be a subflow run that triggered

Ok. So if I

restart

that flow, I would expect the state to be

cached

?

I'm not sure how to get to the subflow page.

Technically, there are no subflows in Prefect 1.0. You just navigate to your flow page of the respective (child) flow

Clicking on the "task" that amounts to the started flow run, and running 'restart' there, doesn't seem to indicate any cache behavior.

Restarts are intended to be used when something fails, not to rerun any arbitrary task when needed.

So if I

restart

that flow, I would expect the state to be

cached

?

Caching is a separate feature and it applies to tasks, not to (sub)flows and it's used to avoid expensive recomputation. If you need more examples for the flow-of-flows, I recommend looking at some examples here and also create_flow_run examples.

Ok. So when you set an idempotency key; if a subsequent run sees that it exists, that flow will simply not be run -- and it will return as a result, the result of the idempotent flow?

Yes that’s right. So if you want a re-trigger of the subflow, you need something like caching the subflow tasks so they don’t re-run + a new idempotency key