Danny Vilela
01/06/2022, 8:24 PMFoo
that maps over a task input (list
) with 3 values. Sometimes, though, some of those values aren’t applicable (for example, if it’s a requested date that doesn’t exist in our database), so I thought of raising prefect.engine.signals.SKIP
to note that (1) this dynamic task input doesn’t apply but also that (2) it wasn’t a failure, per se.
That said, I’m noticing that a task Bar
that is directly downstream of Foo
is also skipping because when Foo
skips any of its mapped tasks. It seems this is intended, but is there a trigger I should raise to note that “it’s fine if any number of these mapped tasks fails”? Bar
has other upstream tasks but I wouldn’t want those to be considered.
Does skip_on_upstream_skip
apply here? Should I configure Bar
such that skip_on_upstream_skip=False
? From the docs here.John Jacoby
01/06/2022, 8:56 PMJohn Jacoby
01/06/2022, 8:57 PMKevin Kho
always_run
. skip_on_upstream_skip
applies to the direct upstream. For each mapped task, it would be the preceding mapped task.Danny Vilela
01/07/2022, 1:09 AM