Hi there. I’m evaluating Prefect for our ETL jobs ...
# ask-community
g
Hi there. I’m evaluating Prefect for our ETL jobs that are getting increasingly complex. I have a group of tasks that can run independently of each other. After they’re all finished, I’d like to run another task, even if any of the tasks in the group has failed. What is the recommended way of achieving this effect in Prefect?
z
Yeah just run
final_task.submit(wait_for=[allow_failure(t) for t in upstream_futures])
g
Cool. I’m going to try it now. Thanks.
z
Where upstream futures is
Copy code
upstream_futures = []
upstream_futures.append(indepent_task_1.submit(...))
upstream_futures.append(indepent_task_2.submit(...))
g
Yup. I’ve got this part already. Didn’t know about the
allow_failure
bit. Thanks!