Thread
#prefect-server
    p

    Payam Vaezi

    11 months ago
    Hello team! Trying to run a workflow with local dask executor, it works fine locally with prefect core and distributing over multiple threads or processes in my local machine over mapped tasks, but when I want to run the same workflow over prefect server, the prefect server runs those mapped tasks in sequential manner. Is this is an expected behavior? Or am I doing something wrong?
    Kevin Kho

    Kevin Kho

    11 months ago
    Hey @Payam Vaezi , maybe it will help is you explicitly specify the number of threads for the local dask executor?
    flow.executor = LocalDaskExecutor(scheduler="threads", num_workers=8)
    p

    Payam Vaezi

    11 months ago
    Thanks for the quick response, let me try that.
    I tried explicitly defining those kwargs, but it seems prefect job disregards those values for some reason, and mapped tasks runs sequentially. Any idea what else may be wrong?
    Kevin Kho

    Kevin Kho

    11 months ago
    Could you show me your flow code if it’s simple enough to share? I do have an idea
    p

    Payam Vaezi

    11 months ago
    Thanks for follow up, just realized it was a user error on my side, once found that executor attribute added to correct flow object it was working fine. Thanks again for timely support.