sometimes fails.
When I go to retry the failing flow, I must rerun
task_a
before running
task_b
.
What is the correct way of doing it in Prefect 2? I read about
refresh_cache
and
persist_result
but I am not sure
b
Bianca Hoch
09/04/2024, 1:54 PM
Hi Mattia, thanks for your question! Just to clarify, you want to rerun
task_a
every time you retry the flow, regardless of the inputs to the task staying the same?
m
Mattia Sappa
09/04/2024, 2:00 PM
Hello Bianca, yes exactly.
In this case,
task_a
is installing dependencies that are needed by
task_b
. But since I ran this in AWS ECS, those dependencies need to be reinstalled every time the flow is retried
c
Chris White
09/04/2024, 3:42 PM
Hey Mattia - if you set
persist_result=False
on
task_a
, it should always rerun (which means it won't ever cache itself); I know this isn't the answer you're looking for, but we explicitly designed for this pattern in Prefect 3.0 with transactions:
Copy code
with transaction():
task_a()
task_b()
^^^ will ensure that both tasks always rerun together, or, if they both complete successfully, stay cached together
upvote 1
m
Mattia Sappa
09/04/2024, 4:02 PM
Thank you Chris! This works for my current simple use case but I will definitely move to t_ransaction_ once I have migrated to Prefect 3 👀 😉
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.