Hi! I have Prefect server & UI running locally (

prefect server start

) and my flows are executing via the agent (

prefect agent start

). I was wondering how I could get them to run on Dask? I have

dask-scheduler

and a few

dask-worker

processes running. When I run my flow using

flow.run(executor=DaskExecutor(address="<tcp://xxx:8786>"))

everything works well, but when I run the flow via prefect server’s UI it doesn’t run on Dask. What am I missing? I tried running the server and agent with

export PREFECT__ENGINE__EXECUTOR__DEFAULT_CLASS="prefect.engine.executors.DaskExecutor"

and

export PREFECT__ENGINE__EXECUTOR__DASK__ADDRESS="<tcp://xxx:8786>"

set but this didn’t help.

Hi tom! This is a common confusion that we are hoping to make more intuitive in the very near future. In particular, for “orchestrated” flow runs that use a Prefect backend, you currently need to configure an Execution Environment that will be used. The Execution environment is where things like choice of executor can be configured. In your case (dask executor + external dask cluster), there’s a recipe right here that should get you up and running: https://docs.prefect.io/orchestration/execution/local_environment.html#examples

Thanks! It worked! I was positive I tried this before but must have done something wrong.

I ran into the exact same issue and worked it out according to the comments here. Also wanted to note that Execution environment seems to have been deprecated in favor of run_configs.

with Flow("...", run_config=LocalRun(), executor=executor) as flow: