Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Whats the process to start the prefect server using a mounted volume? are there ore details on how to use the --use-volume command? How to setup a sample postgres mounted volume etc?

I created a postgres volume using `docker run --rm  --name pg-docker -e POSTGRES_PASSWORD=postgres -d -p 5432:5432 -v $HOME/docker/volumes/postgres:/var/lib/postgresql/data postgres`

but erroring out while using
`prefect server start --use-volume --volume-path $HOME/docker/volumes/postgres:/var/lib/postgresql/data` 

`ERROR: for cli_postgres_1 Cannot create container for service postgres: invalid mode: /var/lib/postgresql/data`

`ERROR: for postgres Cannot create container for service postgres: invalid mode: /var/lib/postgresql/data`
`ERROR: Encountered errors while bringing up the project.` 

I also tried creating a common volume using
`docker volume create pgdata`
Then spinning up postgres with `docker run --rm  --name pg-docker -e POSTGRES_PASSWORD=postgres -d -p 5432:5432 -v pgdata postgres` 
and then starting prefect server with
`prefect server start --use-volume --volume-path pgdata`
Got this exception
`Named volume "pgdata:/var/lib/postgresql/data:rw" is used in service "postgres" but no declaration was found in the volumes section.`
`Exception caught; killing services (press ctrl-C to force)`
`ERROR: Named volume "pgdata:/var/lib/postgresql/data:rw" is used in service "postgres" but no declaration was found in the volumes section.`
`Traceback (most recent call last):`
 `File "/Users/mkorad/.pyenv/versions/3.8.5/lib/python3.8/site-packages/prefect/cli/server.py", line 324, in start`
  `subprocess.check_call(`
 `File "/Users/mkorad/.pyenv/versions/3.8.5/lib/python3.8/subprocess.py", line 364, in check_call`
  `raise CalledProcessError(retcode, cmd)`
`subprocess.CalledProcessError: Command '['docker-compose', 'pull']' returned non-zero exit status 1.` 

Hey <@U0193D2NES2> -- if you do `--use-volume` we will create the volume for you (via docker-compose) and it defaults to `'~/.prefect/pg_data'` -- you can provide a custom path to the data using `--volume-path`

The path set by `--volume-path` is used in the `docker-compose` file to map it into the container <https://github.com/PrefectHQ/prefect/blob/e751130efcd1f44bd9eef4d5a58670beeace00aa/src/prefect/cli/docker-compose.yml#L18>

Thanks Michael, if i use the the default volume `'~/.prefect/pg_data'`  wouldn't it be overwritten during server restarts? Also trying it out :)

great was able to see the state saved by use --use-volume Thanks!

Just a clarifying question, we have situations where our box is replaced during deployments, in that case can the right way to use volumes would be
`prefect server start --volume-path=&lt;&lt;s3-path&gt;&gt;`
Can I use an external path like s3? Does the above command look right for using a custom volume path?

I don't think that docker-compose supports S3 paths

You could sync the S3 path to your machine on startup / teardown or mount the S3 bucket to your local file system