Hey @Marvin how can I make the flow continue even some of the tasks failed?

Here's how you can make your flow continue even if some tasks fail in Prefect: 1. Use

submit

method for tasks: When using the

submit

method, failing tasks can run without stopping the execution of subsequent tasks. This is useful in asynchronous flows. 2. Task Retry Mechanism: Configure the

retry_delay

and

max_retries

parameters in the

@task

decorator to retry failing tasks. 3. Handle exceptions explicitly: You can handle exceptions within the task itself to manage failures more gracefully. Here's an example:

python
from prefect import task, flow
from prefect.task_runners import SequentialTaskRunner

@task
def always_fails_task():
    raise ValueError("I am a bad task")

@task
def always_succeeds_task():
    return "foo"

@flow(task_runner=SequentialTaskRunner())
def example_flow():
    failed_task = always_fails_task.submit()
    successful_task = always_succeeds_task.submit(wait_for=[failed_task])

example_flow()

For more details, you can refer to these resources: 1. Documentation on handling failed tasks: https://docs.prefect.io/latest/concepts/flows/#exception-handling 2. GitHub Issues discussing continued flow execution: - https://github.com/PrefectHQ/prefect/issues/14810 - https://github.com/PrefectHQ/prefect/issues/12804 If you encounter further issues or need more specific use-cases, let me know!