https://prefect.io logo
Join Slack
Powered by
Hello community :slightly_smiling_face: We experie...
# ask-community
t
Hello community 🙂 We experienced a strange behaviour of the "restart" button on prefect cloud: When going to a task run and restarting the task, it correctly reschedules the task, but it doesn't reschedule the downstream tasks (in the UI they stay green instead of light blue)
k
Only Failed tasks are re-scheduled with the restart button. We know there are cases where you want to re-run upstream/downstream so for Orion (Prefect 2.0) this will be supported first-class with subflows being logical units that you can restart together
t
Ah ok. So the fact that it used to work was just luck ? 🙂 Then maybe you should update the message in the UI when clicking the restart button to specify that only the failed/not_successful downstream dependents will be re-run
k
Ah ok we can add a ticket in the UI repo for that
Do you know what current one was?
t
yep:
Click on confirm to restart the flow run _flow_run_name_ from this task run. This will restart your flow run and re-run task _task_name_ and its downstream dependents.
k
Thank you!
@Marvin open “Edit Message of Restart Button for More Clarity” in UI
m
t
Hello @Kevin Kho, to follow on this topic: Given what you told me, I manually put to "failed" the downstream tasks that I want to re-run, however prefect only restart the first task, see the screenshot.
k
How did you manually mark them as failed? Through the UI?
I can try this in a bit and if it fails, we might need to use caching to get the behavior right.
I tested this and it worked for me. Here is my Flow:
Copy code
from prefect import task, Flow
import prefect
import datetime

@task
def task_one():
    <http://prefect.context.logger.info|prefect.context.logger.info>(datetime.datetime.now())
    return 1

@task
def task_two(x):
    <http://prefect.context.logger.info|prefect.context.logger.info>(datetime.datetime.now())
    return x+1

@task
def task_three(x):
    <http://prefect.context.logger.info|prefect.context.logger.info>(datetime.datetime.now())
    return x+1

@task
def task_four(x):
    <http://prefect.context.logger.info|prefect.context.logger.info>(datetime.datetime.now())
    return x+1

with Flow("fail_propagate") as flow:
    task_four(task_three(task_two(task_one())))
I ran this. It was all successful and then i marked tasks 2, 3 and 4 as failed. I then marked the flow run as failed and restarted. Let me see what happens if I don’t mark the flow run as failed
It behaves the same whether or not I mark the Flow run as Failed. Looking more
My current thinking right now is that even if I can replicate the behavior you see, a fix won’t be so straightforward. I would suggest you try caching maybe and then you can start a new flow run without repeating the work. Have you seen caching ? You just need to invalidate the cache when you want to re-run
t
• Yes I marked them as Failed using the UI • I haven't looked at caching yet, will do thanks 🙂 • Something else that I noticed, after marking the tasks as failed: ◦ if a restart the flow from the task (so using the restart button on the task-run page) then it doesn't work ◦ if a restart the flow from the flow-run page and pressing restart, it works correctly
k
Ah yeah because I think restarting from the task run page is just for the task, and from the flow page is for the Flow
You should be good then right?
t
yep, but if this behaviour is expected, the message in the UI is again misleading 🙂
k
yeah the issue for that has been opened
Open in Slack
PreviousNext
Powered by