https://prefect.io logo
Title
m

Miguel

03/23/2021, 5:25 PM
Hello everyone, I'm trying to execute a flow using an existent dask cluster but I'm getting the following error:
[2021-03-23 17:16:53+0000] ERROR - prefect.my-flow | Unexpected error occured in FlowRunner: ModuleNotFoundError("No module named 'prefect'")
This is what I'm doing:
executor = DaskExecutor("my-dask-cluster-ip")
flow.run(executor=executor)
If I use other executors (LocalExecutor or LocalDaskExecutor) it runs fine. Any idea on what I can do to address this issue?
m

Mariia Kerimova

03/23/2021, 5:29 PM
Hello Miguel! Did you register your flow using
flow.register(project_name="<name_of_project">)
?
m

Miguel

03/23/2021, 5:43 PM
No, let me try that. I was running it directly with flow.run as per the docs here: https://docs.prefect.io/core/advanced_tutorials/dask-cluster.html
Can't I run it without the orchestrator?
m

Mariia Kerimova

03/23/2021, 6:34 PM
Yes, you can use run if you spin up your Dask cluster locally. Can you share you redacted/simplified flow, I would help to debug it?
m

Miguel

03/23/2021, 7:13 PM
My dask cluster is remote šŸ˜•
This is what I was trying
import prefect
from prefect.executors import DaskExecutor

@prefect.task
def my_task():
    return 1

with prefect.Flow("my-flow") as flow:
    my_task()

executor = DaskExecutor("external-ip:8786")
flow.run(executor=executor)
I think I've figured it out.. I guess I just had to install prefect in the dask cluster (or any other dependencies the flow depends on)
m

Mariia Kerimova

03/23/2021, 10:11 PM
Sorry for late reply, I’m glad you figured this out! šŸ‘