Hi all, How can I handle the failure in task so that the whole flow doesn't fail because of that when it's being called in a mapped mode? For example, when a shell task is being called to run multiple commands in parallel using

.map

, the whole flow fails immediately if one of the child tasks fail. I have tried passing

return_state=True

but it doesn't seem be effective in mapped mode.

The whole flow shouldn’t fail immediately if one of your task fails. Which version of Prefect are you in? In the following example, the rest of the tasks still run when one task failed

from prefect import task, flow


@task
def task1(num: int):
    if num == 2:
        raise ValueError
    return num


@task
def task2(num: int):
    return num + 1


@flow
def mapped_flow():
    futures = task1.map(list(range(5)))
    return task2.map(futures)


mapped_flow()

Output:

10:27:36.103 | INFO    | Task run 'task1-0' - Finished in state Completed()
10:27:36.109 | INFO    | Task run 'task1-3' - Finished in state Completed()
10:27:36.224 | INFO    | Task run 'task1-1' - Finished in state Completed()
10:27:36.227 | INFO    | Task run 'task1-4' - Finished in state Completed()
10:27:36.456 | INFO    | Task run 'task2-0' - Finished in state Completed()
10:27:36.490 | ERROR   | Task run 'task1-2' - Finished in state Failed('Task run encountered an exception: ValueError\n')
10:27:36.581 | INFO    | Task run 'task2-4' - Finished in state Completed()
10:27:36.589 | INFO    | Task run 'task2-1' - Finished in state Completed()
10:27:36.624 | INFO    | Task run 'task2-3' - Finished in state Completed()
10:27:36.871 | ERROR   | Flow run 'tan-marmoset' - Finished in state Failed('1/5 states are not final.')