@Marvin in prefect 2.X, how can I call a subflow even if some tasks failed?

In Prefect 2.x, you can call a subflow even if some tasks fail by handling the task failures gracefully. To do this, you can use the

result(raise_on_failure=False)

method, which will allow the flow to continue running even if a task encounters an error. Here's a snippet illustrating this concept:

python
from prefect import flow, task

@task
def always_fails_task():
    raise ValueError("I fail successfully")

@task
def always_succeeds_task():
    print("I'm fail safe!")
    return "success"

@flow
def always_succeeds_flow():
    x = always_fails_task.submit().result(raise_on_failure=False)
    y = always_succeeds_task.submit(wait_for=[x])
    return y

if __name__ == "__main__":
    always_succeeds_flow()

In this example,

always_fails_task

will fail, but

always_succeeds_task

will still run successfully because the

result

method was called with

raise_on_failure=False

. You can find more details and context in the Prefect 2.x documentation on flows. Isn't it nice when things just work, even with a hiccup or two? Just like life, flowing along despite minor bumps!

@Marvin will this work using .map?

Using

.map()

in Prefect 2.x can indeed handle scenarios where tasks might fail, just like when using

.result(raise_on_failure=False)

. However, it requires some additional considerations. When you use the

.map()

function, it’s like calling a task multiple times in parallel, and handling failures in this parallelized context can be trickier. Here’s an approach you might take: 1. Handle Individual Task Failures: Within the mapped task, you can handle exceptions so that each individual task can continue or return a default value if it fails. This way, the

.map()

function can complete without stopping prematurely due to failed tasks. 2. Use

result(raise_on_failure=False)

: Just as with individual tasks, you can ensure that a mapped task doesn't raise an exception to the calling flow by using

.result(raise_on_failure=False)

. For more theoretical explanations and code examples, you might want to explore the Prefect 2.x documentation and see how mapping can be used effectively: Prefect's Mapping Documentation. There's nothing like watching tasks fail spectacularly in parallel... and your flow confidently shrugging it off!