https://prefect.io logo
Join Slack
Powered by
Hi <@ULVA73B9P>, how to make the event feed more r...
# marvin-ai
p
Hi @Marvin, how to make the event feed more robust. Currently, it only show the work pool event. I want to know about all event
m
✅ thought for 319.8 seconds
We can definitely show more than just work pool events. A few quick checks usually fix this: Quick UI checks - Reset filters: On the Events page, click Clear filters (or remove any Event type/Resource filters in the pill bar). Make sure the time range is broad enough (e.g., Last 7 or 30 days). - Right workspace: Verify you’re in the workspace where your flows actually run. Verify events are being emitted - Stream what your workspace is sending right now:
Copy code
prefect events stream --run-once
- If you then start a flow, you should see 
prefect.flow-run.*
(and later 
prefect.task-run.*
) events in the stream. - Emit a test event to confirm ingestion and UI display:
Copy code
prefect events emit "test.event" --resource-id "marvin-test-123"
You should see this appear in the Event Feed almost immediately. Common reasons you only see work pool events - Filters/time window are hiding other events (most common): Reset filters and widen the time range. - No flows ran in the selected window: Only system events (like work pool updates) would appear. Kick off a flow run and refresh. - Self-hosted server config is limiting visibility: - Ensure event streaming and persistence are enabled before you load the Events page:
Copy code
PREFECT_SERVER_EVENTS_STREAM_OUT_ENABLED=true
    PREFECT_SERVER_SERVICES_EVENT_PERSISTER_ENABLED=true
- If you’re behind a reverse proxy, make sure WebSockets to 
/api/events/out
are allowed; otherwise you’ll only see a short backfill when the page loads and miss live updates. - Historical window and retention can be tightened by settings; defaults that affect visibility: - Backfill window used by the Event Feed: 
PREFECT_SERVER_EVENTS_MAXIMUM_WEBSOCKET_BACKFILL
(default ~15m) - Retention: 
PREFECT_SERVER_EVENTS_RETENTION_PERIOD
(default ~7 days) Ensure your flows actually emit events - In Prefect 3.x, flow and task state changes emit events automatically; nothing special to enable. You can also send custom events from code if you want to test:
Copy code
from prefect import flow
  from prefect.events import emit_event

  @flow
  def demo():
      emit_event(
          event="demo.completed",
          resource={"prefect.resource.id": "demo-1"},
          payload={"note": "hello from events"}
      )

  if __name__ == "__main__":
      demo()
If you still only see work pool events after these steps, could you share: - Are you on Prefect Cloud or self-hosted? - Prefect version (assuming 3.x)? - A screenshot or description of your current Event Feed filters/time range? - The output of 
prefect events stream --run-once
while you start a flow run? Helpful links - Events concept &amp; structure - Events REST API (read/filter) - Event-driven deployments (example) - Settings reference (server/OSS): https://docs.prefect.io/v3/api-ref/settings-ref
p
Copy code
version: '3.9'
services:
  postgres:
    image: 'postgres:14-alpine'
    environment:
      POSTGRES_USER: '${POSTGRES_USER:-prefect}'
      POSTGRES_PASSWORD: '${POSTGRES_PASSWORD:-prefect_change_me}'
      POSTGRES_DB: '${POSTGRES_DB:-prefect}'
    volumes:
      - 'postgres_data:/var/lib/postgresql/data'
    healthcheck:
      test:
        - CMD-SHELL
        - 'pg_isready -U ${POSTGRES_USER:-prefect} -d ${POSTGRES_DB:-prefect}'
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped
  redis:
    image: 'redis:7-alpine'
    command:
      - redis-server
      - '--appendonly'
      - 'yes'
    volumes:
      - 'redis_data:/data'
    healthcheck:
      test:
        - CMD
        - redis-cli
        - ping
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped
  prefect-server:
    image: 'prefecthq/prefect:3-latest'
    command: "sh -c \" prefect server database upgrade -y && prefect server start --host 0.0.0.0 --no-services \"\n"
    ports:
      - '4200:4200'
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      PREFECT_API_DATABASE_CONNECTION_URL: 'postgresql+asyncpg://${POSTGRES_USER:-prefect}:${POSTGRES_PASSWORD:-prefect_change_me}@postgres:5432/${POSTGRES_DB:-prefect}'
      PREFECT_SERVER_API_HOST: 0.0.0.0
      PREFECT_SERVER_API_PORT: 4200
      PREFECT_REDIS_MESSAGING_HOST: redis
      PREFECT_REDIS_MESSAGING_PORT: 6379
      PREFECT_MESSAGING_BROKER: prefect_redis.messaging
      PREFECT_MESSAGING_CACHE: prefect_redis.messaging
      PREFECT_API_URL: '<http://10.101.3.91:4200/api>'
    healthcheck:
      test:
        - CMD
        - python
        - '-c'
        - "import urllib.request as u; u.urlopen('<http://localhost:4200/api/health>', timeout=1)"
      interval: 20s
      timeout: 10s
      retries: 5
      start_period: 30s
    restart: unless-stopped
  prefect-services:
    image: 'prefecthq/prefect:3-latest'
    command: 'prefect server services start'
    depends_on:
      prefect-server:
        condition: service_healthy
    environment:
      PREFECT_API_DATABASE_CONNECTION_URL: 'postgresql+asyncpg://${POSTGRES_USER:-prefect}:${POSTGRES_PASSWORD:-prefect_change_me}@postgres:5432/${POSTGRES_DB:-prefect}'
      PREFECT_REDIS_MESSAGING_HOST: redis
      PREFECT_REDIS_MESSAGING_PORT: 6379
      PREFECT_API_URL: '<http://10.101.3.91:4200/api>'
    restart: unless-stopped
  init-prefect:
    image: 'prefecthq/prefect:3-latest'
    depends_on:
      prefect-server:
        condition: service_healthy
    environment:
      PREFECT_API_URL: '<http://prefect-server:4200/api>'
    entrypoint: "sh -c \" echo 'Initializing Prefect...' && sleep 5 && prefect work-pool create default-pool --type process || true && prefect work-pool create ocr-pool --type docker --overwrite || true && echo 'Initialization complete.' \"\n"
    restart: 'no'
  workers:
    image: 'ghcr.io/aidevakasha/outlet-scrape:latest'
    command: 'prefect worker start --pool default-pool --type process --limit 5'
    depends_on:
      prefect-server:
        condition: service_healthy
    environment:
      PREFECT_API_URL: '<http://prefect-server:4200/api>'
      PREFECT_LOGGING_LEVEL: INFO
    restart: unless-stopped
  ocr-workers:
    image: 'prefecthq/prefect:3-latest'
    command: "sh -c \"pip install 'prefect[docker]' &&\n      prefect worker start --pool ocr-pool --type docker --limit 5\"\n"
    depends_on:
      prefect-server:
        condition: service_healthy
    environment:
      PREFECT_API_URL: '<http://prefect-server:4200/api>'
      PREFECT_LOGGING_LEVEL: DEBUG
    volumes:
      - '/var/run/docker.sock:/var/run/docker.sock'
    restart: unless-stopped
volumes:
  postgres_data: null
  redis_data: null
  ocr_downloads: null
  ocr_outputs: null
why my selfhosted prefect event feed less verbosed that local prefect run by python? `````` @Marvin
m
✅ thought for 86.6 seconds
Thanks for sharing your compose file — that helps a lot. The short answer: your self‑hosted stack likely isn’t streaming/persisting most events due to how the services are wired and/or how the UI connects, so you’re only seeing a trickle (like work pool events). Local “run by python” tends to look more verbose because everything is running in one process with a working event stream. Here’s a focused checklist to get full event verbosity. 1) Make sure the services can actually talk to your API - In your compose, 
prefect-services
uses 
PREFECT_API_URL: <http://10.101.3.91:4200/api>
. From a Docker container, that host IP may not be routable (hairpin/NAT). - Use the Docker DNS name instead so services hit the API reliably:
Copy code
# In BOTH prefect-services and workers
  PREFECT_API_URL: <http://prefect-server:4200/api>
- I’d also remove or align 
PREFECT_API_URL
in the 
prefect-server
container (it does not need to point at the host IP). Using the same internal URL is safest:
Copy code
PREFECT_API_URL: <http://prefect-server:4200/api>
2) Ensure event streaming/persistence are enabled and running - You start the API with 
--no-services
and a separate 
prefect-services
container — that’s fine, but the services must be healthy and connected. - Add these (explicit) env vars:
Copy code
# In prefect-server (API container)
  PREFECT_SERVER_EVENTS_STREAM_OUT_ENABLED=true
  PREFECT_SERVER_EVENTS_MAXIMUM_WEBSOCKET_BACKFILL=PT60M   # optional: widen initial backfill
  PREFECT_SERVER_EVENTS_RETENTION_PERIOD=P7D               # default; adjust if needed

  # In prefect-services (services container)
  PREFECT_SERVER_SERVICES_EVENT_PERSISTER_ENABLED=true
- Then check logs to confirm the event persister is running:
Copy code
docker logs <prefect-services-container> | grep -i event
You should NOT see repeated connection errors to the API. 3) Verify the UI WebSocket is connected - In your browser DevTools → Network → WS, you should see a connection to 
/api/events/out
with 101 Switching Protocols and it should remain “Open”. - If it immediately closes or fails, you’ll only see a short backfill and miss most live flow/task events. If you’re behind any proxy, ensure WebSockets to 
/api/events/out
are allowed. 4) Quick functional tests - From any container where 
PREFECT_API_URL
points to `http://prefect-server:4200/api`:
Copy code
prefect events emit "test.event" --resource-id "marvin-test-123"
  prefect events stream --run-once
- You should see your test event in the Event Feed and in the stream output. - Kick off a flow run; the stream should show 
prefect.flow-run.Running
and then 
prefect.flow-run.Completed
(and 
prefect.task-run.*
as tasks change state). 5) Keep versions aligned - You’re using 
prefecthq/prefect:3-latest
in some services and a custom worker image elsewhere. Pin a specific 3.x version across all images to avoid protocol drift, e.g.:
Copy code
image: prefecthq/prefect:3.1.9
And ensure your custom image uses the same Prefect version. Suggested compose tweaks (key parts only) ``` services: prefect-server: image: 'prefecthq/prefect:3.1.9' command: "sh -c \"prefect server database upgrade -y && prefect server start --host 0.0.0.0 --no-services\"" environment: PREFECT_API_DATABASE_CONNECTION_URL: 'postgresql+asyncpg://${POSTGRES_USER prefect}${POSTGRES_PASSWORD:-prefect_change_me}@postgres:5432/${POSTGRES_DB:-prefect}' PREFECT_SERVER_API_HOST: 0.0.0.0 PREFECT_SERVER_API_PORT: 4200 PREFECT_REDIS_MESSAGING_HOST: redis PREFECT_REDIS_MESSAGING_PORT: 6379 PREFECT_MESSAGING_BROKER: prefect_redis.messaging PREFECT_MESSAGING_CACHE: prefect_redis.messaging PREFECT_API_URL: 'http://prefect-server:4200/api' PREFECT_SERVER_EVENTS_STREAM_OUT_ENABLED: "true" PREFECT_SERVER_EVENTS_MAXIMUM_WEBSOCKET_BACKFILL: "PT60M" prefect-services: image: 'prefecthq/prefect:3.1.9' command: 'prefect server services start'
environment: PREFECT_API_DATABASE_CONNECTION_URL: 'postgresql+asyncpg://${POSTGRES_USER prefect}${POSTGRES_PASSWORD:-prefect_change_me}@postgres:5432/${POSTGRES_DB:-prefect}' PREFECT_REDIS_MESSAGING_HOST: redis PREFECT_REDIS_MESSAGING_PORT: 6379 PREFECT_API_URL: 'http://prefect-server:4200/api' PREFECT_SERVER_SERVICES_EVENT_PERSISTER_ENABLED: "true" workers: environment: PREFECT_API_URL: 'http://prefect-server:4200/api' ocr-workers: environment: PREFECT_API_URL: 'http://prefect-server:4200/api' ``` If after these changes you still only see work pool events: - Please share: - Output of 
prefect events stream --run-once
while you start a flow run - A snippet from 
docker logs prefect-services
around “event” lines - Whether the 
/api/events/out
WebSocket is connected in your browser Helpful references - Events overview - Settings reference (server events, retention, backfill) - CLI help we verified: 
prefect events --help
(supports 
stream
and 
emit
)
p
version: "3.9" services: postgres: image: "postgres:14-alpine" environment: POSTGRES_USER: "${POSTGRES_USER:-prefect}" POSTGRES_PASSWORD: "${POSTGRES_PASSWORD:-prefect_change_me}" POSTGRES_DB: "${POSTGRES_DB:-prefect}" volumes: - "postgres_data:/var/lib/postgresql/data" healthcheck: test: - CMD-SHELL - "pg_isready -U ${POSTGRES_USER:-prefect} -d ${POSTGRES_DB:-prefect}" interval: 10s timeout: 5s retries: 5 restart: unless-stopped redis: image: "redis:7-alpine" command: - redis-server - "--appendonly" - "yes" volumes: - "redis_data:/data" healthcheck: test: - CMD - redis-cli - ping interval: 10s timeout: 5s retries: 5 restart: unless-stopped prefect-server: image: "prefecthq/prefect:3-latest" command: "sh -c \" prefect server database upgrade -y && prefect server start --host 0.0.0.0 --no-services \"\n" ports: - "4200:4200" depends_on: postgres: condition: service_healthy redis: condition: service_healthy environment: PREFECT_API_DATABASE_CONNECTION_URL: "postgresql+asyncpg://${POSTGRES_USER prefect}${POSTGRES_PASSWORD:-prefect_change_me}@postgres:5432/${POSTGRES_DB:-prefect}" PREFECT_SERVER_API_HOST: 0.0.0.0 PREFECT_SERVER_API_PORT: 4200 PREFECT_REDIS_MESSAGING_HOST: redis PREFECT_REDIS_MESSAGING_PORT: 6379 PREFECT_MESSAGING_BROKER: prefect_redis.messaging PREFECT_MESSAGING_CACHE: prefect_redis.messaging PREFECT_SERVER_EVENTS_STREAM_OUT_ENABLED: true PREFECT_SERVER_EVENTS_MAXIMUM_WEBSOCKET_BACKFILL: PT60M PREFECT_SERVER_EVENTS_RETENTION_PERIOD: P7D _# PREFECT_API_URL: "http://10.101.3.91:4200/api"_ PREFECT_API_URL: http://localhost:4200/api healthcheck: test: - CMD - python - "-c" - "import urllib.request as u; u.urlopen('http://localhost:4200/api/health', timeout=1)" interval: 20s timeout: 10s retries: 5 start_period: 30s restart: unless-stopped prefect-services: image: "prefecthq/prefect:3-latest" command: "prefect server services start" depends_on: prefect-server: condition: service_healthy environment: PREFECT_API_DATABASE_CONNECTION_URL: "postgresql+asyncpg://${POSTGRES_USER prefect}${POSTGRES_PASSWORD:-prefect_change_me}@postgres:5432/${POSTGRES_DB:-prefect}" PREFECT_REDIS_MESSAGING_HOST: redis PREFECT_REDIS_MESSAGING_PORT: 6379 PREFECT_API_URL: http://prefect-server:4200/api PREFECT_SERVER_SERVICES_EVENT_PERSISTER_ENABLED: true restart: unless-stopped init-prefect: image: "prefecthq/prefect:3-latest" depends_on: prefect-server: condition: service_healthy environment: PREFECT_API_URL: "http://prefect-server:4200/api" entrypoint: "sh -c \" echo 'Initializing Prefect...' && sleep 5 && prefect work-pool create default-pool --type process || true && prefect work-pool create ocr-pool --type docker --overwrite || true && echo 'Initialization complete.' \"\n" restart: "no" # workers: # image: 'ghcr.io/aidevakasha/outlet-scrape:latest' # command: 'prefect worker start --pool default-pool --type process --limit 5' _# depends_on:_ # prefect-server: _# condition: service_healthy_ # environment: _# PREFECT_API_URL: 'http://prefect-server:4200/api'_ _# PREFECT_LOGGING_LEVEL: INFO_ # restart: unless-stopped # ocr-workers: # image: 'prefecthq/prefect:3-latest' # command: "sh -c \"pip install 'prefect[docker]' &&\n prefect worker start --pool ocr-pool --type docker --limit 5\"\n" _# depends_on:_ # prefect-server: _# condition: service_healthy_ # environment: _# PREFECT_API_URL: 'http://prefect-server:4200/api'_ _# PREFECT_LOGGING_LEVEL: DEBUG_ # volumes: # - '/var/run/docker.sock:/var/run/docker.sock' # restart: unless-stopped volumes: postgres_data: null redis_data: null ocr_downloads: null ocr_outputs: null @Marvin run this prefect events emit "test.event" --resource-id "marvin-test-123", event not show in the ui
m
✅ thought for 54.6 seconds
Got it — thanks for the compose. The “emit” not showing in the UI almost always comes down to one of these in self‑hosted: 1) The CLI is pointing at the wrong API (often Cloud or no API), so your event goes elsewhere 2) The event services can’t reach your API due to an invalid URL (angle brackets, host mismatch) 3) The UI WebSocket isn’t connected, so you never see live events I see two likely issues in your file: - Angle brackets in URLs: you have values like 
<http://prefect-server:4200/api>
and in the healthcheck 
<http://localhost:4200/api/health>
. Those angle brackets are Slack formatting; if they make it into your compose env, Prefect will treat them literally and fail to connect. - Mixed URLs across containers: 
prefect-server
uses 
PREFECT_API_URL=<http://localhost:4200/api>
while 
prefect-services
uses 
<http://prefect-server:4200/api>
. Standardize to the Docker DNS name for all containers. Do this: A) Fix the compose (remove angle brackets, quote URLs, unify API URL) - Update only the lines shown here; everything else can stay as-is.
Copy code
prefect-server:
  environment:
    PREFECT_API_URL: "<http://prefect-server:4200/api>"
    PREFECT_SERVER_EVENTS_STREAM_OUT_ENABLED: "true"
    PREFECT_SERVER_EVENTS_MAXIMUM_WEBSOCKET_BACKFILL: "PT60M"
    PREFECT_SERVER_EVENTS_RETENTION_PERIOD: "P7D"
  healthcheck:
    test:
      - CMD
      - python
      - -c
      - "import urllib.request as u; u.urlopen('<http://localhost:4200/api/health>', timeout=1)"

prefect-services:
  environment:
    PREFECT_API_URL: "<http://prefect-server:4200/api>"
    PREFECT_SERVER_SERVICES_EVENT_PERSISTER_ENABLED: "true"
B) Restart clean
Copy code
docker compose down -v
docker compose up -d
C) Verify the event pipeline - From your host, explicitly point the CLI at your server and emit a test event:
Copy code
export PREFECT_API_URL=<http://localhost:4200/api>
prefect events emit "test.event" --resource-id "marvin-test-123"
prefect events stream --run-once
You should see the test event printed. If not, the CLI isn’t reaching your server. - Check logs for the services:
Copy code
docker logs <prefect-services-container> | grep -i event
docker logs <prefect-server-container> | grep -i "POST /api/events"
You should see the persister service start and successful POSTs to /api/events. D) Check the UI WebSocket - Open the Events page, then in browser devtools → Network → WS, confirm a live connection to /api/events/out. If it fails/closes, you’ll only see a short backfill and miss live events (proxy must allow WebSockets). E) Sanity: time window and filters - Clear filters on the Events page, set time to “Last 30 minutes” or more, then emit again. If it’s still not showing, could you share: - The output of 
prefect events stream --run-once
right after you emit - A few lines around “event” from 
docker logs prefect-services
- Whether the WS to 
/api/events/out
shows as “Open” in the browser Helpful refs - Events overview: https://docs.prefect.io/v3/concepts/events - Settings (server/events): https://docs.prefect.io/v3/api-ref/settings-ref - CLI we verified: 
prefect events --help
(supports 
emit
and 
stream
)
p
version: "3.9" services: postgres: image: "postgres:14-alpine" environment: POSTGRES_USER: "${POSTGRES_USER:-prefect}" POSTGRES_PASSWORD: "${POSTGRES_PASSWORD:-prefect_change_me}" POSTGRES_DB: "${POSTGRES_DB:-prefect}" volumes: - "postgres_data:/var/lib/postgresql/data" healthcheck: test: - CMD-SHELL - "pg_isready -U ${POSTGRES_USER:-prefect} -d ${POSTGRES_DB:-prefect}" interval: 10s timeout: 5s retries: 5 restart: unless-stopped redis: image: "redis:7-alpine" command: - redis-server - "--appendonly" - "yes" volumes: - "redis_data:/data" healthcheck: test: - CMD - redis-cli - ping interval: 10s timeout: 5s retries: 5 restart: unless-stopped prefect-server: image: "prefecthq/prefect:3-latest" command: "sh -c \" prefect server database upgrade -y && prefect server start --host 0.0.0.0 \"\n" ports: - "4200:4200" depends_on: postgres: condition: service_healthy redis: condition: service_healthy environment: PREFECT_API_DATABASE_CONNECTION_URL: "postgresql+asyncpg://${POSTGRES_USER prefect}${POSTGRES_PASSWORD:-prefect_change_me}@postgres:5432/${POSTGRES_DB:-prefect}" PREFECT_SERVER_API_HOST: 0.0.0.0 PREFECT_SERVER_API_PORT: 4200 PREFECT_REDIS_MESSAGING_HOST: redis PREFECT_REDIS_MESSAGING_PORT: 6379 PREFECT_MESSAGING_BROKER: prefect_redis.messaging PREFECT_MESSAGING_CACHE: prefect_redis.messaging PREFECT_SERVER_EVENTS_STREAM_OUT_ENABLED: true PREFECT_SERVER_EVENTS_MAXIMUM_WEBSOCKET_BACKFILL: PT60M PREFECT_SERVER_EVENTS_RETENTION_PERIOD: P7D _# PREFECT_API_URL: "http://10.101.3.91:4200/api"_ PREFECT_API_URL: http://localhost:4200/api healthcheck: test: - CMD - python - "-c" - "import urllib.request as u; u.urlopen('http://localhost:4200/api/health', timeout=1)" interval: 20s timeout: 10s retries: 5 start_period: 30s restart: unless-stopped # prefect-services: # image: "prefecthq/prefect:3-latest" # command: "prefect server services start" _# depends_on:_ # prefect-server: _# condition: service_healthy_ # environment: _# PREFECT_API_DATABASE_CONNECTION_URL: "postgresql+asyncpg://${POSTGRES_USER prefect}${POSTGRES_PASSWORD:-prefect_change_me}@postgres:5432/${POSTGRES_DB:-prefect}"_ _# PREFECT_REDIS_MESSAGING_HOST: redis_ _# PREFECT_REDIS_MESSAGING_PORT: 6379_ _# PREFECT_API_URL: http://prefect-server:4200/api_ _# PREFECT_SERVER_SERVICES_EVENT_PERSISTER_ENABLED: true_ # restart: unless-stopped init-prefect: image: "prefecthq/prefect:3-latest" depends_on: prefect-server: condition: service_healthy environment: PREFECT_API_URL: "http://prefect-server:4200/api" entrypoint: "sh -c \" echo 'Initializing Prefect...' && sleep 5 && prefect work-pool create default-pool --type process || true && prefect work-pool create ocr-pool --type docker --overwrite || true && echo 'Initialization complete.' \"\n" restart: "no" # workers: # image: 'ghcr.io/aidevakasha/outlet-scrape:latest' # command: 'prefect worker start --pool default-pool --type process --limit 5' _# depends_on:_ # prefect-server: _# condition: service_healthy_ # environment: _# PREFECT_API_URL: 'http://prefect-server:4200/api'_ _# PREFECT_LOGGING_LEVEL: INFO_ # restart: unless-stopped # ocr-workers: # image: 'prefecthq/prefect:3-latest' # command: "sh -c \"pip install 'prefect[docker]' &&\n prefect worker start --pool ocr-pool --type docker --limit 5\"\n" _# depends_on:_ # prefect-server: _# condition: service_healthy_ # environment: _# PREFECT_API_URL: 'http://prefect-server:4200/api'_ _# PREFECT_LOGGING_LEVEL: DEBUG_ # volumes: # - '/var/run/docker.sock:/var/run/docker.sock' # restart: unless-stopped volumes: postgres_data: null redis_data: null ocr_downloads: null ocr_outputs: null without the sevice, the event showing @Marvin
m
✅ thought for 46.3 seconds
Great — that confirms the root cause: when you ran with split services (API + separate services), the event persister wasn’t successfully talking to your API, so emitted events never made it to the DB/stream. When you switched to the monolithic server (API + services in one process), events started showing because the event services are co-located and correctly configured by default. Why the split setup didn’t show events - Mismatched/invalid API URL between containers - Using localhost inside containers or Slack-style angle brackets 
<...>
will break connectivity. - Your API container was set to 
PREFECT_API_URL: <http://localhost:4200/api>
while services used 
<http://prefect-server:4200/api>
. The former is not reachable from other containers; the latter is correct. - Services not enabled or failing to start - The event persister must be enabled and able to reach the API DB and Redis. - WebSocket/backfill looked fine, but without persistence there’s nothing to show. Two good paths forward A) Keep monolithic mode (simplest) - Your current compose runs 
prefect server start
(no --no-services). This runs the API and all services together; since events show now, you’re good. B) If you want split mode, use a minimal, known-good wiring - Key rules: - Use the Docker DNS name everywhere: 
<http://prefect-server:4200/api>
- Quote all URL env vars; do not include angle brackets - Enable event services explicitly - Ensure versions are pinned consistently (avoid 3-latest drift) Here’s a trimmed split-mode example you can drop in (only the relevant parts shown):
Copy code
services:
  prefect-server:
    image: "prefecthq/prefect:3.1.9"
    command: "sh -c \"prefect server database upgrade -y && prefect server start --host 0.0.0.0 --no-services\""
    ports:
      - "4200:4200"
    environment:
      PREFECT_API_DATABASE_CONNECTION_URL: "postgresql+asyncpg://${POSTGRES_USER:-prefect}:${POSTGRES_PASSWORD:-prefect_change_me}@postgres:5432/${POSTGRES_DB:-prefect}"
      PREFECT_SERVER_API_HOST: "0.0.0.0"
      PREFECT_SERVER_API_PORT: "4200"
      PREFECT_REDIS_MESSAGING_HOST: "redis"
      PREFECT_REDIS_MESSAGING_PORT: "6379"
      PREFECT_MESSAGING_BROKER: "prefect_redis.messaging"
      PREFECT_MESSAGING_CACHE: "prefect_redis.messaging"
      PREFECT_API_URL: "<http://prefect-server:4200/api>"
      PREFECT_SERVER_EVENTS_STREAM_OUT_ENABLED: "true"
      PREFECT_SERVER_EVENTS_MAXIMUM_WEBSOCKET_BACKFILL: "PT60M"
      PREFECT_SERVER_EVENTS_RETENTION_PERIOD: "P7D"

  prefect-services:
    image: "prefecthq/prefect:3.1.9"
    command: "prefect server services start"
    depends_on:
      prefect-server:
        condition: service_healthy
    environment:
      PREFECT_API_DATABASE_CONNECTION_URL: "postgresql+asyncpg://${POSTGRES_USER:-prefect}:${POSTGRES_PASSWORD:-prefect_change_me}@postgres:5432/${POSTGRES_DB:-prefect}"
      PREFECT_REDIS_MESSAGING_HOST: "redis"
      PREFECT_REDIS_MESSAGING_PORT: "6379"
      PREFECT_API_URL: "<http://prefect-server:4200/api>"
      PREFECT_SERVER_SERVICES_EVENT_PERSISTER_ENABLED: "true"
Diagnostics to use in split mode - From the services container, verify it can see the API:
Copy code
docker exec -it <prefect-services> sh -lc "prefect config view | grep PREFECT_API_URL"
  docker exec -it <prefect-services> sh -lc "curl -sf <http://prefect-server:4200/api/health> && echo OK"
- Check logs for the event persister starting and posting events:
Copy code
docker logs <prefect-services> | egrep -i "event persister|events|error|traceback"
  docker logs <prefect-server> | grep "POST /api/events" -n
- Functional check:
Copy code
export PREFECT_API_URL=<http://localhost:4200/api>
  prefect events emit "test.event" --resource-id "marvin-test-123"
  prefect events stream --run-once
If you’re happy with the monolithic setup (events are showing), you can stick with it — it’s the easiest way to self-host. If you want, I can turn your compose into a cleaned-up split-mode version and we can validate it step-by-step.
10 Views
Open in Slack
PreviousNext
Powered by