How do I configure a flow to upload local python modules to S3? For example my directory structure looks like this:

project/
    flow.py
    scripts/
        util.py

And in flow.py, there is this import statement:

from scripts.util import Util

Also in flow.py, using this storage:

with Flow('some-name', storage=S3(bucket='some-bucket')):
    ...

But when run from the cloud, it complains

No module named "scripts.util"

If local agent, you can specify a working directory like the third example here. If you use a containerized environment, you really need to package and install them. You need the files available in the container and available in the Python path. None of the storages can inherently install these additional files. I have a packaging article here if it helps. Could we trim the parent message a bit when you get the chance? I think we don’t need the last error

Using a local agent

You can do this then:

flow.run_config = LocalRun(working_dir="/path/to/working-directory")

Or start the agent in the place that has access to those

Do you have any other ideas? When registering a flow, I like that everything in the flow is snapshotted. Meaning even if I change the file on disk, what's in the cloud will keep running as it was when the flow was registered. I would prefer to keep that pattern. And I would like to keep code organized into modules. The modules are not used in any other projects, so it seems overkill to package them.

We don’t hold code like that so we can’t rollback versions either. I think Prefect 2.0 might eventually solve this because you can pin to a git branch/commit for a deployment but that hasn’t been rolled out yet

Okay. Still thinking on this...

I think your options are either Docker or Git. The current Git storage does let you pin to a commit though I think