3 years and maybe I am missing something. If a task fails in a flow, should the whole flow exit? As far as my 100 other flows work, if a task fails, that seems to be OK, as I used allow_failure(failed_task_result) to continue on. This does not make any sense if a task failure exits the flow.

yes

If a task fails in a flow, should the whole flow exit?

by default

In [17]: from prefect import flow, task

In [18]: @task
    ...: def foo():
    ...:     raise ValueError()
    ...:

In [19]: @flow
    ...: def my_flow():
    ...:     foo()
    ...:

In [20]: my_flow()
12:13:53.889 | INFO    | prefect.engine - Created flow run 'deft-seahorse' for flow 'my-flow'
12:13:54.838 | INFO    | Flow run 'deft-seahorse' - Created task run 'foo-0' for task 'foo'
12:13:54.841 | INFO    | Flow run 'deft-seahorse' - Executing 'foo-0' immediately...
12:13:55.215 | ERROR   | Task run 'foo-0' - Encountered exception during execution:
Traceback (most recent call last):
....
  File "<ipython-input-18-54d755c6ce79>", line 3, in foo
    raise ValueError()
ValueError
12:13:55.492 | ERROR   | Flow run 'deft-seahorse' - Finished in state Failed('Flow run encountered an exception. ValueError\n')

however if you call your task with

return_state=True

, then you can avoid letting a task failure interrupt your flow

In [21]: @flow
    ...: def my_flow():
    ...:     foo_final_state = foo(return_state=True)
    ...:     return foo_final_state.is_failed()
    ...:

In [22]: my_flow()
12:20:50.256 | INFO    | prefect.engine - Created flow run 'cordial-bandicoot' for flow 'my-flow'
12:20:51.394 | INFO    | Flow run 'cordial-bandicoot' - Created task run 'foo-0' for task 'foo'
12:20:51.396 | INFO    | Flow run 'cordial-bandicoot' - Executing 'foo-0' immediately...
12:20:51.703 | ERROR   | Task run 'foo-0' - Encountered exception during execution:
Traceback (most recent call last):
....
  File "<ipython-input-33-54d755c6ce79>", line 3, in foo
    raise ValueError()
ValueError
12:20:51.880 | ERROR   | Task run 'foo-0' - Finished in state Failed('Task run encountered an exception: ValueError\n')
12:20:51.995 | INFO    | Flow run 'cordial-bandicoot' - Finished in state Completed()
Out[22]: True

Hey thanks for this. Maybe I always made everything so reliable it never happened. Much appreciated.