https://prefect.io logo
t

tom

10/08/2020, 5:27 PM
Hi! I have Prefect server & UI running locally (
prefect server start
) and my flows are executing via the agent (
prefect agent start
). I was wondering how I could get them to run on Dask? I have
dask-scheduler
and a few
dask-worker
processes running. When I run my flow using
flow.run(executor=DaskExecutor(address="<tcp://xxx:8786>"))
everything works well, but when I run the flow via prefect server’s UI it doesn’t run on Dask. What am I missing? I tried running the server and agent with
export PREFECT__ENGINE__EXECUTOR__DEFAULT_CLASS="prefect.engine.executors.DaskExecutor"
and
export PREFECT__ENGINE__EXECUTOR__DASK__ADDRESS="<tcp://xxx:8786>"
set but this didn’t help.
c

Chris White

10/08/2020, 6:16 PM
Hi tom! This is a common confusion that we are hoping to make more intuitive in the very near future. In particular, for “orchestrated” flow runs that use a Prefect backend, you currently need to configure an Execution Environment that will be used. The Execution environment is where things like choice of executor can be configured. In your case (dask executor + external dask cluster), there’s a recipe right here that should get you up and running: https://docs.prefect.io/orchestration/execution/local_environment.html#examples
t

tom

10/08/2020, 6:30 PM
Thanks! It worked! I was positive I tried this before but must have done something wrong.
s

Suchindra

02/18/2021, 2:08 AM
I ran into the exact same issue and worked it out according to the comments here. Also wanted to note that Execution environment seems to have been deprecated in favor of run_configs.
Copy code
with Flow("...", run_config=LocalRun(), executor=executor) as flow: