https://prefect.io logo
#prefect-community
Title
# prefect-community
j

Jacques

04/27/2020, 2:25 PM
I want to do
trigger=all_successful
for the mapped function, but it only works for unmapped tasks
k

Kyle Moon-Wright

04/27/2020, 3:29 PM
Hey Jacques, How are you using triggers with your mapped task? Keep in mind whenever a mapped task is reduced by a downstream task, the trigger function will be applied to all of the mapped children. For example, if a reducing task as an 
all_successful
 task but one of the mapped tasks fails, the reducing task’s trigger will fail. Note that triggers are only applied on the children tasks, not the parent.
j

Jacques

04/27/2020, 4:41 PM
Not sure I follow 100%, in my case it is the task feeding the mapped task that fails
and then the mapped task fails because it tries to process the error result from the first task
k

Kyle Moon-Wright

04/27/2020, 5:07 PM
Hmm perhaps I’m misunderstanding, the mapping task’s trigger will apply to all of the mapped children tasks and any failures that occur will bubble to the parent task. However if you wanted to change the conditions of the trigger, you can use other trigger functions which are documented here. What was your expected behavior for your use case?
j

Jacques

04/27/2020, 6:12 PM
So I'm doing this
Copy code
extract_dummy = ExtractDummyTask(trigger=all_successful)
transform_dummy = TransformDummyTask(trigger=all_successful)
load_dummy = LoadDummyTask(trigger=all_successful)
# Create dummy flow
with Flow("dummy_flow") as flow:

    # Extract
    values = extract_dummy()

    # Transform
    transformed_values = transform_dummy.map(data=values)

    # Load
    load_dummy(data=transformed_values)
But if I make ExtractDummyTask raise a RuntimeError() then TransformDummyTask tries to run using the RuntimeError - this error specifically
Task 'TransformDummyTask': unexpected error while running task: TypeError("Cannot map over unsubscriptable object of type <class 'RuntimeError'>: RuntimeErr...")
load has the expected behaviour though:
Task 'LoadDummyTask': finished task run for task with final state: 'TriggerFailed'
k

Kyle Moon-Wright

04/27/2020, 6:40 PM
Hey @Jacques, let me look into this and get back to you.
j

Jacques

04/27/2020, 6:50 PM
Thanks, appreciated!
k

Kyle Moon-Wright

04/27/2020, 7:23 PM
Hey @Jacques, after asking internally this looks to be a bug - the mapped task shouldn’t attempt to iterate if the upstream task fails. Would it be possible to open an issue on github with a minimal reproducible example? I can help if needed.
j

Jacques

04/27/2020, 8:26 PM
Sure, give me a few mins to put something together
Hope that is good enough
👍 1
k

Kyle Moon-Wright

04/27/2020, 8:51 PM
That’s great, thank you!