Hello everyone, I'm trying to execute a flow using...
# prefect-serverm
Miguel
03/23/2021, 5:25 PMHello everyone, I'm trying to execute a flow using an existent dask cluster but I'm getting the following error:
Copy code
[2021-03-23 17:16:53+0000] ERROR - prefect.my-flow | Unexpected error occured in FlowRunner: ModuleNotFoundError("No module named 'prefect'") Copy code
executor = DaskExecutor("my-dask-cluster-ip")
flow.run(executor=executor)m
Mariia Kerimova
03/23/2021, 5:29 PMHello Miguel! Did you register your flow using
flow.register(project_name="<name_of_project">)m
Miguel
03/23/2021, 5:43 PMNo, let me try that. I was running it directly with flow.run as per the docs here: https://docs.prefect.io/core/advanced_tutorials/dask-cluster.html
Miguel
03/23/2021, 5:46 PMCan't I run it without the orchestrator?
m
Mariia Kerimova
03/23/2021, 6:34 PMYes, you can use run if you spin up your Dask cluster locally. Can you share you redacted/simplified flow, I would help to debug it?
m
Miguel
03/23/2021, 7:13 PMMy dask cluster is remote š
Miguel
03/23/2021, 7:43 PMThis is what I was trying
Copy code
import prefect
from prefect.executors import DaskExecutor
@prefect.task
def my_task():
return 1
with prefect.Flow("my-flow") as flow:
my_task()
executor = DaskExecutor("external-ip:8786")
flow.run(executor=executor)Miguel
03/23/2021, 7:53 PMI think I've figured it out.. I guess I just had to install prefect in the dask cluster (or any other dependencies the flow depends on)
m
Mariia Kerimova
03/23/2021, 10:11 PMSorry for late reply, Iām glad you figured this out! š
2 Views