https://prefect.io logo
s

Sophia Ponte

08/03/2023, 9:39 PM
Hi all -- I have a prefect flow with 3 sequential tasks (A -> B -> C). If one of my tasks fails, is there a way to restart the flow from the failed task in the UI? For example, if task B fails, is there a way to restart the flow from B, so that it progresses to C? Rather than restarting from A?
n

Nate

08/03/2023, 10:23 PM
hi @Sophia Ponte - yes! you just need to `persist_results=True` on your flow! you can optionally set something like
retries=2
in your
flow
decorator as well, and prefect will know that persisting these results is necessary in order to retry note that if you're doing remote execution, you should have something like an S3 or GCS block to set as the
result_storage
on the flow
s

Sophia Ponte

08/03/2023, 10:30 PM
Thanks for the tip! But how do I actually restart the flow from the failed task on the UI? For our use case, we cannot do it automatically by configuring retries. With persistence set to true, will restarting the flow (via the retry button) skip the already completed tasks?
n

Nate

08/03/2023, 10:36 PM
yes, if you have results persisted, when you click the retry button, the flow should start from the first failed task
s

Sophia Ponte

08/03/2023, 11:02 PM
Thanks again. One more question: In prefect 1, there was functionality to change the state of a task. Then, when we would restart the flow, the flow would start from the next task, rather than the one that failed. For for the (A -> B -> C) example, if B fails, we would set it's state to "success", and then restart the flow, and C would execute. In Prefect 2, I tried the same process, but it didn't work as I expected. That is, I configured
persist_results
and
result_storage
, then when B failed, I set it's status to "Completed" then retried the flow. It started from B again, rather than starting from the next task, C. Is this the expected behavior?
n

Nate

08/03/2023, 11:29 PM
how did you do this?
I set it's status to "Completed" then retried the flow
s

Sophia Ponte

08/03/2023, 11:31 PM
In the failed task's page of the UI, I click on the 3 dots on the top right hand side of the page and click "Change State". Then fill this out and click "submit"
n

Nate

08/03/2023, 11:34 PM
ok thanks - i will have time in a bit to doublecheck my prior here on expected behaviour and make sure nothing is broken
s

Sophia Ponte

08/03/2023, 11:40 PM
Thanks! No rush. To be precise, here's what I observed: • if I restart the flow without modifying the failed task's state, the flow restarts with the failed task. • if I modify the failed task's state to be "Completed", and then retry the flow, the flow reruns the failed task (and fails again). I would instead expect the flow to understand that the task was manually edited to completed, and move on to the following task.
Hi @Nate -- do you have any update on this?
n

Nate

08/16/2023, 6:11 PM
hi @Sophia Ponte - sorry i havent had time to look into this deeply. would you be able to open an issue for this?
s

Sophia Ponte

08/16/2023, 6:12 PM
Sure. Is this the right spot for UI issues? https://github.com/PrefectHQ/prefect/issues
n

Nate

08/16/2023, 6:12 PM
yep!
thank you 1
thanks!
s

sjammula

08/23/2023, 7:49 PM
@Sophia Ponte can you please post the ticket here which you created,since we are running into the same issue where we need this fix asap as it is a major blocker for our org. cc: @Nate
m

Matan Kleyman

09/18/2023, 2:34 PM
@Sophia Ponte @Nate Hey! Unfortunately, we are running into the same issue. Any updates on that? Thanks!
s

Sophia Ponte

09/18/2023, 4:47 PM
Hey -- unfortunately I didn't get around to creating the issue until just now. Here is the link: https://github.com/PrefectHQ/prefect/issues/10749
m

Matan Kleyman

09/18/2023, 7:11 PM
@Sophia Ponte Thanks. I commented as well. Did you find a workaround to skip tasks ? Becuase its pretty frustating since im not able to skip any failed tasks.
s

Sophia Ponte

09/18/2023, 7:42 PM
Can you just encode the behavior into your flows? Like put a try/except block around your failing tasks to handle them and allow your overall flow to continue? If you need to be able to handle it from the UI side, then I don't have any workarounds.
2 Views