I am trying to test caching behavior. I've run a flow to completion, and then marked it as failed, so that I can restart it. I have a log that looks like this:

13:53:26 INFO CloudFlowRunner 
    Flow run SUCCESS: all reference tasks succeeded
13:55:13 INFO
    User marked flow run 
            as Failed
13:55:21 INFO
    null restarted this flow run
13:55:22
    INFO agent Submitted for execution: Container ID: 
        <id>
13:55:22 INFO
    S3 Downloading flow from s3://<key>
13:55:23 INFO
    S3 Flow successfully downloaded. ETag: <>, LastModified: <> VersionId: <>
13:55:23 INFO CloudFlowRunner
    Beginning Flow run for <pipeline>
13:55:23 INFO CloudFlowRunner
    Flow run SUCCESS: all reference tasks succeeded

I would expect the flow to re-run entirely. However, it doesn't look like it does (it just jumps straight into Success). Is this due to Prefect's default input caching? To be specific, this flow is a flow-of-flows. I have two subflows

I think what is happening if you mark the flow as failed, but a restart specifically re-schedules all tasks in a Failed state, and it seems like you don’t have any?

the easiest way would be to just create a new flow run since it seems you want to rerun the whole flow rather than only Failed task runs to explain it a little more: restarting from the UI doesn't create a new flow run, instead it updates the history of the "old" flow run by rerunning the flow run from a Failed task. If you would like to rerun the flow from a specific task, you would need to mark this specific task as Failed and then click Restart from Failed (from this specific task)

But note that a sub-flow with the same idempotency key doesn’t re-run. it just returns the already existing run