Hey all, I'm probably making this much harder than it should be, but how do I restart a flow in the UI with the exact same parameters without any task result cacheing? I've turned checkpointing off on all of my tasks and I've even expanded
merge()
to allow me to pass kwargs to the
Merge
constructor so I can turn off that task checkpointing, but it still picks up in the middle of my flow at the task that failed. I can't have this happen because it requires previously downloaded data referenced in other tasks that get cleaned on failure/success.
My question is: How can I restart a flow in the UI from the beginning with the exact same parameters without any task result caching?
k
Kevin Kho
12/15/2021, 5:13 PM
Hello @Connor Martin, this feature doesn’t exist yet but we have had discussions about providing a dropdown of parameter sets to kick off a new flow run. But in the mean time, we don’t have such a feature
c
Connor Martin
12/15/2021, 5:17 PM
Hey @Kevin Kho thanks for the quick response.
Ahhh so there's no way to restart a flow just as it is from the beginning?
What do you mean by a dropdown of parameter sets?
k
Kevin Kho
12/15/2021, 5:19 PM
A lot of users just want a non technical person to go in and select previously run parameter sets to kick off a new flow run. Think for report generation
Kevin Kho
12/15/2021, 5:25 PM
You can set the states of the tasks from Success to Scheduled maybe and then trigger a restart
c
Connor Martin
12/15/2021, 5:29 PM
Doesn't turning off checkpointing cause the result of a task to no longer be stored? Shouldn't that mean that the flow bubbles back up to the beginning of the dag?
I'll go give setting the state a try as well, thank you for the suggestion!
k
Kevin Kho
12/15/2021, 5:31 PM
But you need to re-register the Flow for turning off checkpointing to apply so it won’t apply on the previous flow run
c
Connor Martin
12/15/2021, 5:32 PM
I have reregistered and manually started a flow run where I set the state of the flow to failed.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.