https://prefect.io logo
s

Sergi de Pablos

01/27/2022, 6:43 AM
Hi everyone. I have a process that submits multiple jobs to an external service, and I have to wait for all of them to be finished before proceeding. I've found how to wait in a task for some external process https://github.com/PrefectHQ/prefect/discussions/3944 to be finished but no idea how to wait for multiple ones to be finished. Thanks
k

Kevin Kho

01/27/2022, 6:50 AM
Isn’t it just a matter of adding this code to the other tasks? Or you can map over tasks with this code? Or am I missing something? 😅
s

Sergi de Pablos

01/27/2022, 7:41 AM
Pretty new to Prefect, so maybe I'm not understanding how everything works. The process is like this: I need to call a external process multiple times on a parallel way, wait for all of them to finish, and then execute another single task over the output of the previous parallel tasks (some files in an S3 folder)
k

Kevin Kho

01/27/2022, 8:45 AM
The mapping docs will help you with that
s

Sergi de Pablos

01/27/2022, 9:51 PM
Exactly what I was looking for @Kevin Kho. Thanks a lot!