Guys, is there a best practice for making sure the same code is available to workers on remote boxes? I've factored out a few functions into a utility python file that I import. I'm running dask-worker manually on a remote server and it complains (obviously) ModuleNotFoundError. I thought copying the code to remote and running worker from that directory might help but no dice

Hey @Kevin Systrom - I believe you’re one step away; if you add that directory to your

$PATH

prior to starting the dask-worker I’d expect you should be good, e.g.,

export PATH=/abs/path/to/directory:$PATH
dask-worker scheduler-address

good call. Always a manager never a sys admin 😉 thanks

haha anytime

@Kevin Systrom maybe @Chris White already addressed everything you need, but we (long time Prefect users at a healthcare startup) have solved this with putting dependencies in docker images and having Dask workers use those. That might be overkill for your current setup, but holler if it's useful for me to tell you more.