Hello everyone, I'm trying to execute a flow using...
# prefect-server
m
Hello everyone, I'm trying to execute a flow using an existent dask cluster but I'm getting the following error:
Copy code
[2021-03-23 17:16:53+0000] ERROR - prefect.my-flow | Unexpected error occured in FlowRunner: ModuleNotFoundError("No module named 'prefect'")
This is what I'm doing:
Copy code
executor = DaskExecutor("my-dask-cluster-ip")
flow.run(executor=executor)
If I use other executors (LocalExecutor or LocalDaskExecutor) it runs fine. Any idea on what I can do to address this issue?
m
Hello Miguel! Did you register your flow using
flow.register(project_name="<name_of_project">)
?
m
No, let me try that. I was running it directly with flow.run as per the docs here: https://docs.prefect.io/core/advanced_tutorials/dask-cluster.html
Can't I run it without the orchestrator?
m
Yes, you can use run if you spin up your Dask cluster locally. Can you share you redacted/simplified flow, I would help to debug it?
m
My dask cluster is remote šŸ˜•
This is what I was trying
Copy code
import prefect
from prefect.executors import DaskExecutor

@prefect.task
def my_task():
    return 1

with prefect.Flow("my-flow") as flow:
    my_task()

executor = DaskExecutor("external-ip:8786")
flow.run(executor=executor)
I think I've figured it out.. I guess I just had to install prefect in the dask cluster (or any other dependencies the flow depends on)
m
Sorry for late reply, Iā€™m glad you figured this out! šŸ‘