Hi all — I have a dynamic task `Foo` that maps ove...
# ask-community
d
Hi all — I have a dynamic task
Foo
that maps over a task input (
list
) with 3 values. Sometimes, though, some of those values aren’t applicable (for example, if it’s a requested date that doesn’t exist in our database), so I thought of raising
prefect.engine.signals.SKIP
to note that (1) this dynamic task input doesn’t apply but also that (2) it wasn’t a failure, per se. That said, I’m noticing that a task
Bar
that is directly downstream of
Foo
is also skipping because when
Foo
skips any of its mapped tasks. It seems this is intended, but is there a trigger I should raise to note that “it’s fine if any number of these mapped tasks fails”?
Bar
has other upstream tasks but I wouldn’t want those to be considered. Does
skip_on_upstream_skip
apply here? Should I configure
Bar
such that
skip_on_upstream_skip=False
? From the docs here.
j
Hi Danny! I've never tried skip_on_upstream_skip, but you might look at setting trigger=any_successful in the decorator. https://docs.prefect.io/api/latest/triggers.html#functions
🙌 1
Or maybe trigger=not_all_skipped
k
John’s suggestion is good here with the trigger being the place to look. You can also do
always_run
.
skip_on_upstream_skip
applies to the direct upstream. For each mapped task, it would be the preceding mapped task.
1
d
I missed this thread — thank you @John Jacoby and @Kevin Kho!! 🙏