Jacques
04/27/2020, 2:25 PMtrigger=all_successful
for the mapped function, but it only works for unmapped tasksKyle Moon-Wright
04/27/2020, 3:29 PMall_successful
task but one of the mapped tasks fails, the reducing task’s trigger will fail. Note that triggers are only applied on the children tasks, not the parent.Jacques
04/27/2020, 4:41 PMKyle Moon-Wright
04/27/2020, 5:07 PMJacques
04/27/2020, 6:12 PMextract_dummy = ExtractDummyTask(trigger=all_successful)
transform_dummy = TransformDummyTask(trigger=all_successful)
load_dummy = LoadDummyTask(trigger=all_successful)
# Create dummy flow
with Flow("dummy_flow") as flow:
# Extract
values = extract_dummy()
# Transform
transformed_values = transform_dummy.map(data=values)
# Load
load_dummy(data=transformed_values)
But if I make ExtractDummyTask raise a RuntimeError() then TransformDummyTask tries to run using the RuntimeError - this error specifically Task 'TransformDummyTask': unexpected error while running task: TypeError("Cannot map over unsubscriptable object of type <class 'RuntimeError'>: RuntimeErr...")
Task 'LoadDummyTask': finished task run for task with final state: 'TriggerFailed'
Kyle Moon-Wright
04/27/2020, 6:40 PMJacques
04/27/2020, 6:50 PMKyle Moon-Wright
04/27/2020, 7:23 PMJacques
04/27/2020, 8:26 PMKyle Moon-Wright
04/27/2020, 8:51 PM