@Marvin Is it possible to end a flow with success even if some tasks fail?

Yes, it's possible to configure a Prefect flow to end successfully even if some tasks fail. This can be achieved by controlling how exceptions are handled and what the flow returns. In Prefect 2.x, you can configure individual task runs to not raise exceptions and thereby not affect the flow's success status. You can do this by setting

raise_on_failure=False

when calling

.submit()

on a task. The flow will complete successfully if it ends up returning a "Completed" future, regardless of the failure of other tasks. Here's an example from the documentation:

python
@flow
def always_succeeds_flow():
    x = always_fails_task.submit().result(raise_on_failure=False)
    y = always_succeeds_task.submit(wait_for=[x])
    return y

if __name__ == "__main__":
    always_succeeds_flow()

In Prefect 3.x, the behavior is similar, but more controlled at the flow level. The flow's final state depends on its return values, not automatically on the state of individual task runs. If any exceptions within the tasks are suppressed (again, using

raise_on_failure=False

), the flow can choose to end in a "Completed" state. Literal values or explicitly returned "Completed" states from the flow will result in a successful flow run. Here's more context: - Flows now complete successfully if they return non-failing states or if exceptions are appropriately suppressed. - You can control this behavior by handling task exceptions carefully. For more details, you can visit these helpful links: - Prefect 2.x Flow Concepts - Prefect 3.x Upgrade Guide "Raise_on_failure=False": saving you from flow anxiety since... well, since now.