when using docker storage and/or remote executors, what’s considered the best practice for shipping code around? does it make sense to rely on cloudpickle to send any required non-third party code via the flow itself, or should everything run from the same base image that has the needed internal functions?

Based on my own experience, I recommend: - putting the code in a common base image - ideally converting the code into an “installable” package so that PATH issues don’t arise the reason is that it’s hard to know apriori what the working directory for the Flow process is (although, we could document this and enforce a common location). Additionally, any code that is imported based on file location will be interpreted by cloudpickle as a module, so that can just cause some headaches with newer users if the flow ends up running in a place with different relative file paths

thanks @Chris White! all our code is already in 1 rather large Docker image that I’m using as my

Storage

base, so for now everything works fine. just trying to think through the case where a worker cluster stays running, and users submit flows that could have slightly different versions of some of the involved functions…safest best is definitely to always rebuild the base image and redeploy everything, but it’d be nice to avoid that in some cases if the changes are really small

oh that makes a lot of sense; I’m sure someone could tell us why this isn’t the best idea, but if your cluster is long-standing and not adaptive, you could exec into the workers and update the code manually. Because of how cloudpickle works, your tasks should actually import the new code (this might warrant a test to make sure)