Hi everyone.
I have a process that submits multiple jobs to an external service, and I have to wait for all of them to be finished before proceeding. I've found how to wait in a task for some external process https://github.com/PrefectHQ/prefect/discussions/3944 to be finished but no idea how to wait for multiple ones to be finished. Thanks
k
Kevin Kho
01/27/2022, 6:50 AM
Isn’t it just a matter of adding this code to the other tasks? Or you can map over tasks with this code? Or am I missing something? 😅
s
Sergi de Pablos
01/27/2022, 7:41 AM
Pretty new to Prefect, so maybe I'm not understanding how everything works. The process is like this: I need to call a external process multiple times on a parallel way, wait for all of them to finish, and then execute another single task over the output of the previous parallel tasks (some files in an S3 folder)
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.