https://prefect.io logo
Title
a

Aniruddha Sengupta

03/08/2022, 1:55 PM
Hello Hope that you are well. I have a parent task I have created and I am able to create a create a series of task runs based on the inputs according to this documentation: https://docs.prefect.io/core/idioms/task-run-names.html. How can I ensure that the children tasks that are created from this are able to run in parallel instead of sequentially? Do I need to set the flow's executor to be the LocalDaskExecutor? Thanks for your help,
k

Kevin Kho

03/08/2022, 2:20 PM
Hi @Aniruddha Sengupta, yes exactly you need to use the LocalDaskExecutor for parallelization
a

Aniruddha Sengupta

03/08/2022, 2:43 PM
Hi Kevin Thanks for your reply. Just to clarify, I have a task called
url_extraction
which takes a
lang
input. When the Flow is then created, it is mapped to a list of languages which creates a series of children tasks based on the language. When I set the Flow's executor like so:
flow.executor = LocalDaskExecutor()
, this will ensure that the children tasks will be executed in parallel? Thanks, Aniruddha
k

Kevin Kho

03/08/2022, 2:48 PM
If they are independent, yes
a

Aniruddha Sengupta

03/08/2022, 2:51 PM
Ok, thanks so much for your help Kevin.
k

Kevin Kho

03/08/2022, 2:51 PM
Of course!
❤️ 1