Really sorry to keep bombarding the channel with s...
# prefect-community
r
Really sorry to keep bombarding the channel with so many questions - I think this will be the last one for the day. My question is: when you dockerize your flow, does this docker image replace the image of your dask workers for the purpose of executing your flow? I was under the impression that it does. However, I am getting this error w/ a custom library when running the flow from Prefect Server.
ModuleNotFoundError: No module named 'alternative_data_pipelines'
Note that the package
alternative_data_pipelines
exists in my docker file and has been pip-installed. No issues with the health checker - everything has been registered successfully and runs ok locally and from the docker container.
j
Hi @Riley Hun the docker image does not automatically replace the image used for your dask workers. If you use something like the DaskKubernetesEnvironment it does due to it spinning up a cluster for each run but otherwise you will have to ensure your dask workers have their needed dependencies installed
r
What about for Dask Gateway?
j
I’m not very familiar with Dask Gateway, maybe the devs over on their repo would be able to help there 🙂
r
I also wasn't able to overcome the
ModuleNotFoundError
. I've installed the custom package to the docker file of the dask workers successfully. I then SSH-ed into the dask workers on GKE and did a
pip list
to see if the package was there, and it was.
r
I am facing a similar problem. I ran dask-workers locally in a directory where I have file
somefile.py
but dask says no module named somefile.