10/08/2020, 5:27 PM
Hi! I have Prefect server & UI running locally (
prefect server start
) and my flows are executing via the agent (
prefect agent start
). I was wondering how I could get them to run on Dask? I have
and a few
processes running. When I run my flow using"<tcp://xxx:8786>"))
everything works well, but when I run the flow via prefect server’s UI it doesn’t run on Dask. What am I missing? I tried running the server and agent with
export PREFECT__ENGINE__EXECUTOR__DEFAULT_CLASS="prefect.engine.executors.DaskExecutor"
export PREFECT__ENGINE__EXECUTOR__DASK__ADDRESS="<tcp://xxx:8786>"
set but this didn’t help.

Chris White

10/08/2020, 6:16 PM
Hi tom! This is a common confusion that we are hoping to make more intuitive in the very near future. In particular, for “orchestrated” flow runs that use a Prefect backend, you currently need to configure an Execution Environment that will be used. The Execution environment is where things like choice of executor can be configured. In your case (dask executor + external dask cluster), there’s a recipe right here that should get you up and running:


10/08/2020, 6:30 PM
Thanks! It worked! I was positive I tried this before but must have done something wrong.


02/18/2021, 2:08 AM
I ran into the exact same issue and worked it out according to the comments here. Also wanted to note that Execution environment seems to have been deprecated in favor of run_configs.
Copy code
with Flow("...", run_config=LocalRun(), executor=executor) as flow: