https://prefect.io
Join Slack
Hello community :slightly_smiling_face: We experienced a strange behaviour of the "restart" button o...
# ask-community
t
Hello community 🙂 We experienced a strange behaviour of the "restart" button on prefect cloud: When going to a task run and restarting the task, it correctly reschedules the task, but it doesn't reschedule the downstream tasks (in the UI they stay green instead of light blue)
k
Only Failed tasks are re-scheduled with the restart button. We know there are cases where you want to re-run upstream/downstream so for Orion (Prefect 2.0) this will be supported first-class with subflows being logical units that you can restart together
t
Ah ok. So the fact that it used to work was just luck ? 🙂 Then maybe you should update the message in the UI when clicking the restart button to specify that only the failed/not_successful downstream dependents will be re-run
k
Ah ok we can add a ticket in the UI repo for that
Do you know what current one was?
t
yep:
Click on confirm to restart the flow run _flow_run_name_ from this task run. This will restart your flow run and re-run task _task_name_ and its downstream dependents.
k
Thank you!
@Marvin open “Edit Message of Restart Button for More Clarity” in UI
m
t
Hello @Kevin Kho, to follow on this topic: Given what you told me, I manually put to "failed" the downstream tasks that I want to re-run, however prefect only restart the first task, see the screenshot.
k
How did you manually mark them as failed? Through the UI?
I can try this in a bit and if it fails, we might need to use caching to get the behavior right.
I tested this and it worked for me. Here is my Flow:
Copy code
from prefect import task, Flow
import prefect
import datetime

@task
def task_one():
    <http://prefect.context.logger.info|prefect.context.logger.info>(datetime.datetime.now())
    return 1

@task
def task_two(x):
    <http://prefect.context.logger.info|prefect.context.logger.info>(datetime.datetime.now())
    return x+1

@task
def task_three(x):
    <http://prefect.context.logger.info|prefect.context.logger.info>(datetime.datetime.now())
    return x+1

@task
def task_four(x):
    <http://prefect.context.logger.info|prefect.context.logger.info>(datetime.datetime.now())
    return x+1

with Flow("fail_propagate") as flow:
    task_four(task_three(task_two(task_one())))
I ran this. It was all successful and then i marked tasks 2, 3 and 4 as failed. I then marked the flow run as failed and restarted. Let me see what happens if I don’t mark the flow run as failed
It behaves the same whether or not I mark the Flow run as Failed. Looking more
My current thinking right now is that even if I can replicate the behavior you see, a fix won’t be so straightforward. I would suggest you try caching maybe and then you can start a new flow run without repeating the work. Have you seen caching ? You just need to invalidate the cache when you want to re-run
t
• Yes I marked them as Failed using the UI • I haven't looked at caching yet, will do thanks 🙂 • Something else that I noticed, after marking the tasks as failed: ◦ if a restart the flow from the task (so using the restart button on the task-run page) then it doesn't work ◦ if a restart the flow from the flow-run page and pressing restart, it works correctly
k
Ah yeah because I think restarting from the task run page is just for the task, and from the flow page is for the Flow
You should be good then right?
t
yep, but if this behaviour is expected, the message in the UI is again misleading 🙂
k
yeah the issue for that has been opened
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by