Mattia Sappa
09/04/2024, 11:26 AMtask_a -> task_b
. task_a
always succeed while task_b
sometimes fails.
When I go to retry the failing flow, I must rerun task_a
before running task_b
.
What is the correct way of doing it in Prefect 2? I read about refresh_cache
and persist_result
but I am not sureBianca Hoch
09/04/2024, 1:54 PMtask_a
every time you retry the flow, regardless of the inputs to the task staying the same?Mattia Sappa
09/04/2024, 2:00 PMtask_a
is installing dependencies that are needed by task_b
. But since I ran this in AWS ECS, those dependencies need to be reinstalled every time the flow is retriedChris White
persist_result=False
on task_a
, it should always rerun (which means it won't ever cache itself); I know this isn't the answer you're looking for, but we explicitly designed for this pattern in Prefect 3.0 with transactions:
with transaction():
task_a()
task_b()
^^^ will ensure that both tasks always rerun together, or, if they both complete successfully, stay cached togetherMattia Sappa
09/04/2024, 4:02 PM