https://prefect.io logo
Title
p

Payam Vaezi

10/20/2021, 6:02 PM
Hello team! Trying to run a workflow with local dask executor, it works fine locally with prefect core and distributing over multiple threads or processes in my local machine over mapped tasks, but when I want to run the same workflow over prefect server, the prefect server runs those mapped tasks in sequential manner. Is this is an expected behavior? Or am I doing something wrong?
k

Kevin Kho

10/20/2021, 6:20 PM
Hey @Payam Vaezi , maybe it will help is you explicitly specify the number of threads for the local dask executor?
flow.executor = LocalDaskExecutor(scheduler="threads", num_workers=8)
p

Payam Vaezi

10/20/2021, 6:26 PM
Thanks for the quick response, let me try that.
I tried explicitly defining those kwargs, but it seems prefect job disregards those values for some reason, and mapped tasks runs sequentially. Any idea what else may be wrong?
k

Kevin Kho

10/20/2021, 7:35 PM
Could you show me your flow code if it’s simple enough to share? I do have an idea
p

Payam Vaezi

10/21/2021, 8:09 PM
Thanks for follow up, just realized it was a user error on my side, once found that executor attribute added to correct flow object it was working fine. Thanks again for timely support.
👍 1