Hi @Marvin, please provide a docker compose file with recommended settings for prefect 3.2

Here's a basic

docker-compose.yaml

file for setting up Prefect 3.2 using Docker. This configuration includes a Prefect worker and connects to Docker for task execution:

yaml
version: '3.8'

services:
  prefect:
    image: prefecthq/prefect:3.2.0
    restart: always
    command: 
      - "prefect"
      - "server"
      - "start"
    ports:
      - "4200:4200" # Prefect UI
    environment:
      - PREFECT_API_URL=<http://localhost:4200>

  worker:
    image: prefecthq/prefect:3.2.0
    restart: always
    command:
      [
        "prefect",
        "worker",
        "start",
        "--pool",
        "docker-work",
        "--install-policy",
        "if-not-present"
      ]
    env_file:
      - .env
    ports:
      - "8081:8081"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - DOCKER_HOST=unix:///var/run/docker.sock

### Key Points: - Prefect Service: Runs the Prefect server, exposing the UI on port 4200. - Worker Service: Configured to use Docker for task execution, exposing port 8081. - Environment Variables: Make sure to configure the

.env

file with necessary environment variables like

PREFECT_API_URL

. For further customization, refer to the Prefect documentation on Docker deployments.

@Marvin can you give similar for podman-compose and also with the basic auth added

Creating a

podman-compose

file for Prefect 3.2 with basic authentication involves similar steps to Docker Compose, but with Podman's nuances. Here's a basic setup, including basic authentication:

yaml
version: '3.8'

services:
  prefect:
    image: prefecthq/prefect:3.2.0
    command: 
      - "prefect"
      - "server"
      - "start"
    ports:
      - "4200:4200" # Prefect UI
    environment:
      - PREFECT_API_URL=<http://localhost:4200>
      - PREFECT_SERVER_API_BASIC_AUTH_ENABLED=true
      - PREFECT_SERVER_API_BASIC_AUTH_USERNAME=your_username
      - PREFECT_SERVER_API_BASIC_AUTH_PASSWORD=your_password

  worker:
    image: prefecthq/prefect:3.2.0
    command:
      [
        "prefect",
        "worker",
        "start",
        "--pool",
        "docker-work",
        "--install-policy",
        "if-not-present"
      ]
    env_file:
      - .env
    ports:
      - "8081:8081"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - DOCKER_HOST=unix:///var/run/docker.sock

### Key Adjustments: - Basic Auth: Enabled with

PREFECT_SERVER_API_BASIC_AUTH_ENABLED=true

and credentials set via

PREFECT_SERVER_API_BASIC_AUTH_USERNAME

and

PREFECT_SERVER_API_BASIC_AUTH_PASSWORD

. - Podman Specifics: Podman is largely compatible with Docker Compose files. Ensure Podman is properly configured to replace Docker where necessary. For further details on using Podman with Prefect, you may need to adapt based on specific Podman configurations or refer to Prefect's deployment documentation.

@Marvin: your docker compose file generates the following error:

Attaching to prefect-1, worker-1
prefect-1  | Exception caught; killing services (press ctrl-C to force)
prefect-1  | Traceback (most recent call last):
prefect-1  |   File "/usr/local/lib/python3.7/site-packages/prefect/cli/server.py", line 632, in start
prefect-1  |     ["docker-compose", "pull"], cwd=compose_dir_path, env=env
prefect-1  |   File "/usr/local/lib/python3.7/subprocess.py", line 358, in check_call
prefect-1  |     retcode = call(*popenargs, **kwargs)
prefect-1  |   File "/usr/local/lib/python3.7/subprocess.py", line 339, in call
prefect-1  |     with Popen(*popenargs, **kwargs) as p:
prefect-1  |   File "/usr/local/lib/python3.7/subprocess.py", line 800, in __init__
prefect-1  |     restore_signals, start_new_session)
prefect-1  |   File "/usr/local/lib/python3.7/subprocess.py", line 1551, in _execute_child
prefect-1  |     raise child_exception_type(errno_num, err_msg, err_filename)
prefect-1  | FileNotFoundError: [Errno 2] No such file or directory: 'docker-compose': 'docker-compose'

The error you're encountering indicates that the

docker-compose

command is not found in your environment. This usually happens if Docker Compose is not installed or not in the system's PATH. ### Solutions: 1. Ensure Docker Compose is Installed: - Verify installation by running

docker-compose --version

in your terminal. - If not installed, follow the installation guide here. 2. Use Podman Compose: - If you're using Podman, ensure

podman-compose

is installed and in your PATH. - Replace

docker-compose

with

podman-compose

in your command if you're using Podman as a Docker alternative. 3. Alternative Execution: - Run the Prefect server and worker directly using Prefect commands if Compose is not an option:

bash
     prefect server start
     prefect worker start --pool docker-work --install-policy if-not-present

If you continue to face issues, ensure your environment variables and paths are correctly set, and consult the Prefect deployment documentation for detailed guidance.

@Marvin

docker-compose

is deprecated, please update your information to the new

docker compose

interface

@mark doerr did you get a docker compose deployment working?

Hi @Jeff S, yes, after many trials I managed with local persitent volumes and an new database - I guess it was a problem of migration (still do not know, why it was complaining about setuptools missing in the container when I used the old docker-compose file with the system persistent volume 🤔