Hi everyone, I have prefect-server set up via minikube and the official helm charts (prefect-server-2021.03.10) including a Kubernetes Agent set up via helm chart. I want a flow to use the DaskExecutor so I've set the flow.executor to an instance of the DaskExecutor. I'm able to register and run the flow, but when I run the flow a job pod starts, but no other pods do where I would expect dask workers to spin up. Does anyone know what is going on here? The simple task and dask worker definitions can be seen at https://github.com/Adam-D-Lewis/sample-flow if needed.

Hi Adam, the issue here is that you're configuring your executor in a section of code that will only run if your flow source is run as a script (within a check like below)

if __name__ == "__main__":
   ...

When prefect loads your flow to execute it, it is not run as a script, so that block won't execute and your executor won't be configured. I recommend only putting the

flow.register

call within that block and moving the rest of the configuration up above that check.

Ahh, I had assumed I only needed to set the DaskExecutor when registering the the flow. I'll give the changes you mention a try. Thanks for the quick response!

If you were using "pickle-based" storage that would be true, but github storage is "script-based", so your flow source is executed during every execution. See https://docs.prefect.io/orchestration/flow_config/storage.html#pickle-vs-script-based-storage for more info on the difference.