I wanted to see an example of parallel workflow fo...
# ask-community
b
I wanted to see an example of parallel workflow for prefect, can anyone share me a link of parallel workflows?
j
In Prefect parallelism can easily be achieved through the use of a Dask executor Example: https://docs.prefect.io/core/advanced_tutorials/dask-cluster.html#the-dask-executor Does this help?
b
a slightly more complex example?
j
I welcome anyone to provide a snippet of code if they have a more complex flow they would be willing to share but, still going by the docs, there’s this advanced tutorial https://docs.prefect.io/core/advanced_tutorials/advanced-mapping.html#advanced-features which uses the Dask executor to perform something slightly more complex
Flow from this example ^
b
can this example be considered as parallel execution?
when I use dask executor that task is being processed parallalely by default?
j
Yes! When using a dask executor this will be executed in parallel. By default if you do not provide a scheduler
address
to the executor it will create one locally for the run of the flow
b
providing address to the Dask Executor makes the task run parallely right?
j
If you have multiple dask workers then it will run tasks in parallel https://distributed.dask.org/en/latest/worker.html
b
how to set for multiple workers? via cli?
j
Yes you can, that dask executor tutorial I linked above has a way for doing it via CLI