Hello all, I am looking through the documentation and I am struggling to find information on passing volume information to Docker Storage. My flow outputs some files to disk and I would like to mount a volume so that we can see the files that were created on the filesystem, outside of the docker container. How would I accomplish this?

I cant speak to how to mount up a certain directory, but in your specific case I would look at potentially saving these files by using an S3 bucket, which you can authenticate regardless of where in a container you are 🙂

Hey @Matthew Blau I believe volume mounting is intended to be encapsulated on the Agent to apply to any submitted flow runs rather than the Storage object of the flow itself, and can be configured to multiple volumes. Of course as mentioned, persisting these files to another db is also viable.

Hey @Matthew Blau 🙂 If you use Docker Agent, you can use

--volume

to mount a volume. However, if you need to persist files I suggest to use S3/Azure Blob Storage, as @Samuel Hinton said

@Kyle Moon-Wright I had seen that option for volume mounting and that it wants /host/path:/container/path. That would mean something like .:/app, correct? Our existing solution with a docker compose file does this that for volume mounting.

volumes:
      - .:/app

The files that are being written are files that are then used to populate our db and having them easily accessible to the file system for checking that the correct info is populated in them is our use case here

I think that will do it, as an example for all submitted flow runs:

prefect agent docker start --volume /source/file:/target/file

Hopefully someone can verify 😅

@Kyle Moon-Wright I have been successful in writing the exact directory that I wanted, but the issue is just that: it wants an exact directory and I run into a module not found error if I have it output files to a different directory the directory that the flow is located in; so it would seem that it does not scale very well for multiple projects that all write to their own directories