https://prefect.io logo
#prefect-community
Title
# prefect-community
a

Anders Segerberg

04/11/2022, 6:03 PM
I am trying to test caching behavior. I've run a flow to completion, and then marked it as failed, so that I can restart it. I have a log that looks like this:
Copy code
13:53:26 INFO CloudFlowRunner 
    Flow run SUCCESS: all reference tasks succeeded
13:55:13 INFO
    User marked flow run 
            as Failed
13:55:21 INFO
    null restarted this flow run
13:55:22
    INFO agent Submitted for execution: Container ID: 
        <id>
13:55:22 INFO
    S3 Downloading flow from s3://<key>
13:55:23 INFO
    S3 Flow successfully downloaded. ETag: <>, LastModified: <> VersionId: <>
13:55:23 INFO CloudFlowRunner
    Beginning Flow run for <pipeline>
13:55:23 INFO CloudFlowRunner
    Flow run SUCCESS: all reference tasks succeeded
I would expect the flow to re-run entirely. However, it doesn't look like it does (it just jumps straight into Success). Is this due to Prefect's default input caching? To be specific, this flow is a flow-of-flows. I have two subflows
k

Kevin Kho

04/11/2022, 6:05 PM
I think what is happening if you mark the flow as failed, but a restart specifically re-schedules all tasks in a Failed state, and it seems like you don’t have any?
upvote 1
a

Anna Geller

04/11/2022, 6:11 PM
the easiest way would be to just create a new flow run since it seems you want to rerun the whole flow rather than only Failed task runs to explain it a little more: restarting from the UI doesn't create a new flow run, instead it updates the history of the "old" flow run by rerunning the flow run from a Failed task. If you would like to rerun the flow from a specific task, you would need to mark this specific task as Failed and then click Restart from Failed (from this specific task)
k

Kevin Kho

04/11/2022, 6:11 PM
But note that a sub-flow with the same idempotency key doesn’t re-run. it just returns the already existing run
👍 1
6 Views