Hi everyone,I am trying to connect prefect to a local dask cluster to test out distributed calculation of a flow that I want to register to a dask cluster in the cloud later.I followed the tutorial https://docs.prefect.io/core/advanced_tutorials/dask-cluster.html and that works great as long as everything is defined in one python file. But if I want to import any function from a second file running it results in a
.Does anybody have any experience on how Dask is handling these imports?
thanks, i will have a look!to answer the question: I haven’t set up any storage. Just running the tutorial file as is and moving one function to another file and then reimporting breaks it.
1 year ago
That makes sense @Clemens - Prefect doesn't do anything special to package your flow and so relies on imports existing in the runtime environment; dependencies should either be pip-installable or present in the environment. As
@Michael Adkins mentioned, this is usually resolved by packaging your flows using whatever storage is appropriate for your setup.