Hey all, happy new year everyone. Hope everyone st...
# prefect-community
w
Hey all, happy new year everyone. Hope everyone stays safe and healthy! I’m currently experimenting with an ETL architecture, and I read in the docs that Dask cluster was the preferred way of running a flow. But I couldn’t find examples of using the Executer within the functional API, only with the imperative API (
flow.run(executor=Dask
). I did however started a Dask Cluster locally, and set the environment variables, as per the docs, but my tasks are still syncronous, and not running in parallel, and I can’t see anything in the dask workers logs. is there anything I should be doing to enjoy the speed and benefits of the parallel and distributed computation?
For anybody who comes across this in the future, I fixed by setting the
executer
and the
environment
like so:
Copy code
python
```executor = DaskExecutor()
environment = LocalEnvironment(executor)

with Flow('Distributed Flow', environment=environment) as flow:
z
Heads up that we’d recommend using the new
RunConfigs
over environments. https://docs.prefect.io/orchestration/flow_config/run_configs.html