Is it possible to retry mapped tasks? Whenever I t...
# ask-community
t
Is it possible to retry mapped tasks? Whenever I try it I get nulls being passed as the mapped input parameter. We are using prefect cloud with prefect results.
k
Hey @Trevor Kramer, are you using caching by chance?
And do you have the results templated to provide unique result locations?
t
I am not using result templates - I didn't think that was necessary with PrefectResults
k
I don’t think it is. I was just wondering if there was already a result seen and the mapped task is skipping cuz of it. I misunderstood. It sounds like the issue is when you restart it doesn’t know the values anymore. Let me look a bit.
So the previous task right before the failed mapped task is persisted with PrefectResult?
t
we set flow.result = PrefectResult() before we register
and don't explicitly set result types anywhere else
k
Ok so I will look more into this tomorrow when the team is around to give you a better answer but at the moment I suspect that this is not working because
flow.result = PrefectResult()
is applied to all tasks as a place to write results, but I think it gets overwritten with each task. If there is a mapped task, I think only one value is saved at a time. When you try to retry from failure, there is only one thing to pull at most for the mapped task (assuming the parameter came from the preceding task).
So I think what would specifically help you is applying a Result to the preceding task so that the result is persisted. That task-specific result will override Prefect result. Also make that task-specific result templated so that each of the map gets a separate result. Now when you retry the task, it knows where to grab those
t
I think the issue is that flow.result doesn't seem to be respected. If I set
Copy code
Flow('pipeline', result=PrefectResult())
then it works as expected.
k
Oh that’s good to know. Thanks for circling back on this! I will explore this and we’ll see if we need to open an issue for this on our end.