Constantino Schillebeeckx
04/29/2022, 2:24 PMFilterTask
with my own filter func, does the task still filter out the default NoResult
None
and exceptions?foo = FilterTask(filter_func=lambda x: isinstance(x, int))
@task
def m1(x):
if x == 5:
raise ValueError()
return x
with Flow('foo') as f:
l1 = m1.map([1, 2, 3, 4, 5])
foo(l1)
Anna Geller
04/29/2022, 3:13 PMflow.set_reference_tasks([your_important_task])
this way, regardless of the end result of the filterTask, you can customize when the flow run is considered SuccessfulConstantino Schillebeeckx
04/29/2022, 3:32 PMThe flow run will be considered a success because foo filters out the failed tasks - does it make sense?that was my question, it seems like in addition to the
filter_func
the task will also filter out exceptionsAnna Geller
04/29/2022, 3:38 PMthat was my question, it seems like in addition to theyup! you're rightthe task will also filter out exceptionsfilter_func
the reference task doesn't work for me because i need to know whether any task in any part of the flow failsGot it. I think state handler seems reasonable to figure that out and get notified. The only downside is: if you have 200 tasks and 20 of them fail, you'll get 20 notifications (which can get annoying), not a single one. If that's OK, then a state handler would be the best option
Constantino Schillebeeckx
04/29/2022, 3:49 PMAnna Geller
04/29/2022, 3:52 PMConstantino Schillebeeckx
04/29/2022, 3:54 PMAnna Geller
04/29/2022, 3:55 PM