I have a flow that has 4 tasks, first being a para...
# ask-communitya
Adam Brusselback
10/12/2021, 6:45 PMI have a flow that has 4 tasks, first being a parameter, second returning an intermediary result, third using that result to get some more intermediary results < fails here, fourth using all these intermediary results to run a query against my database. In testing I caused some failures on the 3rd step, and when trying to restart the failed job the intermediary results from the second step are set as None
Adam Brusselback
10/12/2021, 6:46 PMIf I just start a new run, everything is fine. It's just restarting the failed run that has problems.
k
Kevin Kho
10/12/2021, 6:52 PM
What results do you have set on these tasks?
a
Adam Brusselback
10/12/2021, 6:54 PMJust a simple string (which is a key I use to look up a dict of secrets in my next task)
k
Kevin Kho
10/12/2021, 6:54 PM
I mean Prefect Result
Kevin Kho
10/12/2021, 6:55 PM
If you are restarting a flow run, the results are loaded in and then passed to the next task so that you don’t need to re-run the task
a
Adam Brusselback
10/12/2021, 7:00 PMAh that's my problem, I need to instrument my task with a result type (prefect result should work just fine in this case). I was just surprised it wasn't rerunning all tasks if the result wasn't persisted
Adam Brusselback
10/12/2021, 7:17 PMOn a related note, if I have a task where I don't want to persist the results but I just want it re-run in the event of a flow re-run, is there a way to do that?
k
Kevin Kho
10/12/2021, 7:18 PM
Kind of. You can set the state to Failed or maybe Scheduled through the API or UI.
a
Adam Brusselback
10/12/2021, 7:45 PMIt actually looks like I want to use SecretResult here, as it ensures the code is re-run and I am writing my own code to get my secrets and I don't want it persisted anywhere. So I am changing my task to subclass the SecretBase rather than using the @task annotation
Adam Brusselback
10/12/2021, 7:52 PMVery much appreciate you pointing me in the right direction. Hope I haven't been too much of a pain with these questions over the past weeks.
k
Kevin Kho
10/12/2021, 7:53 PM
Oh that’s news to me. Of course no problem!
3 Views