https://prefect.io logo
#prefect-community
Title
# prefect-community
v

Vikram Iyer

06/30/2020, 8:43 AM
Hi again, I am trying to bring up prefect using the below docker-compose file.
Copy code
version: '3.7'

services:
  ad-ds-outlier-app:
    build: .
    command: >
      gunicorn -c "python:config.gunicorn"
        --reload "<http://myapp.app:create_app()|myapp.app:create_app()>"
    networks:
      - prefect-server
    volumes:
      - '.:/app'
      - './config/settings.py:/app/instance/settings.py'
    ports:
      - 12345:12345

  postgres:
    image: "postgres:11"
    ports:
      - "5432:5432"
    environment:
      POSTGRES_USER: prefect
      POSTGRES_PASSWORD: test-password
      POSTGRES_DB: prefect_server
    networks:
      - prefect-server
    restart: "always"
    command:
      - "postgres"
      # explicitly set max connections
      - "-c"
      - "max_connections=150"

  hasura:
    image: "hasura/graphql-engine:v1.1.0"
    ports:
      - "3000:3000"
    command: "graphql-engine serve"
    environment:
      HASURA_GRAPHQL_DATABASE_URL: "<postgresql://prefect:test-password@postgres:5432/prefect_server>"
      HASURA_GRAPHQL_ENABLE_CONSOLE: "true"
      HASURA_GRAPHQL_SERVER_PORT: "3000"
      HASURA_GRAPHQL_QUERY_PLAN_CACHE_SIZE: 100
    networks:
      - prefect-server
    restart: "always"
    depends_on:
      - postgres

  graphql:
    image: "prefecthq/server:latest"
    ports:
      - "4201:4201"
    command: bash -c "prefect-server database upgrade -y && python src/prefect_server/services/graphql/server.py"
    environment:
      PREFECT_SERVER_DB_CMD: "echo 'DATABASE MIGRATIONS SKIPPED'"
      PREFECT_SERVER__DATABASE__CONNECTION_URL: "<postgresql://prefect:test-password@postgres:5432/prefect_server>"
      PREFECT_SERVER__HASURA__ADMIN_SECRET: hasura-secret-admin-secret
      PREFECT_SERVER__HASURA__HOST: hasura
    networks:
      - prefect-server
    restart: "always"
    depends_on:
      - hasura

  scheduler:
    image: "prefecthq/server:latest"
    command: "python src/prefect_server/services/scheduler/scheduler.py"
    environment:
      PREFECT_SERVER__HASURA__ADMIN_SECRET: hasura-secret-admin-secret
      PREFECT_SERVER__HASURA__HOST: hasura
    networks:
      - prefect-server
    restart: "always"
    depends_on:
      - graphql

  apollo:
    image: "prefecthq/apollo:latest"
    ports:
      - "4200:4200"
    command: "npm run serve"
    environment:
      HASURA_API_URL: <http://hasura:3000/v1alpha1/graphql>
      PREFECT_API_URL: <http://graphql:4201/graphql/>
      PREFECT_API_HEALTH_URL: <http://graphql:4201/health>
      PREFECT_SERVER__TELEMETRY__ENABLED: "true"
    networks:
      - prefect-server
    restart: "always"
    depends_on:
      - graphql

  ui:
    image: "prefecthq/ui:latest"
    ports:
      - "8080:8080"
    command: "/intercept.sh"
    environment:
      PREFECT_SERVER__GRAPHQL_URL: <http://localhost:4200/graphql>
    networks:
      - prefect-server
    restart: "always"
    depends_on:
      - apollo

  db:
    image: 'ellerbrock/alpine-mongodb'
    environment:
      SERVICE_HOST: 0.0.0.0
    volumes:
      - ./db/mongodb/db:/data/db
      - ./entrypoint.sh:/entrypoint.sh
    networks:
      - prefect-server
    ports:
      - 27017:27017

volumes:
  db:

networks:
  prefect-server:
    name: prefect-server
The only additions to the original prefect provided docker-compose are a service and db. When I perform a
flow.register
, I get an error which states,
'project_name' is a required field when using prefect cloud
. I am running this locally and do not know where do I mention to use backend instead of cloud by default. The normal way is on the host I can simply run
prefect backend server; prefect server start
, but since I am using a custom docker-compose file, not sure how to get around this. Any help? 😕
j

josh

06/30/2020, 11:36 AM
@Vikram Iyer On the same machine you are registering your flow on run
prefect backend server
before you call register. It shouldn’t have anything to do with the server’s docker compose. Optionally you could also set the environment variable
Copy code
PREFECT__BACKEND=server
v

Vikram Iyer

06/30/2020, 11:38 AM
So, the machine I am registering my flow on is a container too. So, do I just set the enviroment variable on that container?
j

josh

06/30/2020, 11:41 AM
Yep that should work!
Since you’re working within a Docker container connected to the same network as the server you may need to specifically also set the endpoint of the server
Copy code
PREFECT__SERVER__ENDPOINT=apollo:4200
Or something like that where you want to set the endpoint to the accessible ip/name and port of the apollo container
v

Vikram Iyer

06/30/2020, 4:09 PM
Thanks @josh. 🙂
So, with all these containers up, I am able to register flows. However, to run them from the UI, I will require agents. How do I get an agent docker container running using the docker-compose. For example, I can use
prefect agent start docker
otherwise, but how do I add agent containers via docker-compose.