https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> I am having an issue where the tasks ...
# ask-marvin
b
@Marvin I am having an issue where the tasks within my flow do not register in the prefect UI. The code all executes as expected and the flow completes, but the tasks do not appear to register properly. here is a minimal example that I am running
Copy code
"""Minimal test flow to verify tasks appear in Prefect UI."""

import asyncio

from prefect import flow, task
from prefect.logging import get_run_logger


@task
async def task_one(value: str) -> str:
    """First task - waits 3 seconds."""
    logger = get_run_logger()
    <http://logger.info|logger.info>(f"Task one starting with value: {value}")
    await asyncio.sleep(3)
    <http://logger.info|logger.info>("Task one complete")
    return f"one-{value}"


@task
async def task_two(value: str) -> str:
    """Second task - waits 5 seconds."""
    logger = get_run_logger()
    <http://logger.info|logger.info>(f"Task two starting with value: {value}")
    await asyncio.sleep(5)
    <http://logger.info|logger.info>("Task two complete")
    return f"two-{value}"


@task
async def task_three(val1: str, val2: str) -> str:
    """Third task - combines results, waits 2 seconds."""
    logger = get_run_logger()
    <http://logger.info|logger.info>(f"Task three combining: {val1} + {val2}")
    await asyncio.sleep(2)
    result = f"{val1}|{val2}"
    <http://logger.info|logger.info>(f"Task three complete: {result}")
    return result


@flow(timeout_seconds=60)
async def test_task_visibility_flow(input_value: str = "test") -> str:
    logger = get_run_logger()
    <http://logger.info|logger.info>(f"Flow starting with input: {input_value}")

    # Call tasks with await - these should show in UI
    result_one = await task_one(input_value)
    result_two = await task_two(input_value)
    result_three = await task_three(result_one, result_two)

    <http://logger.info|logger.info>(f"Flow complete with result: {result_three}")
    return result_three


if __name__ == "__main__":
    asyncio.run(test_task_visibility_flow())
m
✅ thought for 410.0 seconds
Thanks for the clear repro. Your flow/task code looks fine — in Prefect 3.x, awaited async tasks inside an async flow should produce Task Runs that show up in the UI. When the flow completes but tasks don’t appear, the most common cause is that the process running your code is not successfully streaming task-state events to the Prefect API via WebSocket. In Prefect 3, flow state is still persisted over HTTP, but task visibility in the UI relies on a WebSocket connection to the API. If that connection can’t be established, your code still runs and the flow completes, but individual Task Runs won’t show up. Quick checks and fixes: 1) Verify you’re connected to an API (Server or Cloud) - Run:
Copy code
prefect config view | egrep 'PREFECT_API_URL|PREFECT_API_KEY|PREFECT_UI_URL'
- If 
PREFECT_API_URL
is empty, point it to your server or Cloud: - Local Server:
Copy code
prefect server start
    export PREFECT_API_URL=<http://localhost:4200/api>
- Prefect Cloud:
Copy code
prefect cloud login  # follow prompts
    # or set
    export PREFECT_API_URL=<https://api.prefect.cloud/api/accounts/<account_id>/workspaces/<workspace_id>>
    export PREFECT_API_KEY=<your_key>
2) Check for WebSocket connectivity issues - Firewalls/proxies often block websockets. Look for warnings by running with debug logs:
Copy code
PREFECT_LOGGING_LEVEL=DEBUG python your_script.py
If you see messages about failing to open an events websocket or “offline events client”, that explains missing Task Runs in the UI. - Ensure your network allows ws/wss to the same host/port as your API (e.g., ws://localhost:4200/api/ws for local server, wss://api.prefect.cloud/api/ws for Cloud). 3) Avoid ephemeral/offline modes for observability - If you’re using an ephemeral/local dev server or running entirely offline, Task Runs will not be visible in the UI. Prefer running against a started Server or Cloud for full task visibility. 4) Your asyncio usage is fine - The pattern
Copy code
if __name__ == "__main__":
      asyncio.run(test_task_visibility_flow())
is the documented way to run async flows locally in Prefect 3.x. It will show tasks as long as the API + websocket connection is healthy. - For reference, see the async examples: Run work concurrently (async) A quick end-to-end test (local server): 1) In one terminal: 
prefect server start
2) In another terminal:
Copy code
export PREFECT_API_URL=<http://localhost:4200/api>
   PREFECT_LOGGING_LEVEL=DEBUG python your_script.py
3) Open the UI and confirm the three task runs appear under the flow run. If you still don’t see tasks: - What does 
prefect version
output? - Are you on Prefect Server or Prefect Cloud? - What’s the output of 
prefect config view | egrep 'PREFECT_API_URL|PREFECT_API_KEY'
? - Do you see any “websocket” or “events client” warnings in DEBUG logs? Share those details and I’ll help you pinpoint the exact cause.
b
@Marvin after submitting a flow to prefect, there is not logging in the prefect API container about websockets or event client. do these logs seem normal
Copy code
(sourcing) ➜  sourcing git:(SOURC-149--baseline-native-sla-flow) ✗ docker compose logs pre
fect-api
prefect-api-1  | 22:23:06.621 | DEBUG   | prefect.profiles - Using profile 'ephemeral'
prefect-api-1  | 
prefect-api-1  |  ___ ___ ___ ___ ___ ___ _____
prefect-api-1  | | _ \ _ \ __| __| __/ __|_   _|
prefect-api-1  | |  _/   / _|| _|| _| (__  | |
prefect-api-1  | |_| |_|_\___|_| |___\___| |_|
prefect-api-1  | 
prefect-api-1  | Configure Prefect to communicate with the server with:
prefect-api-1  | 
prefect-api-1  |     prefect config set PREFECT_API_URL=<http://0.0.0.0:4200/api>
prefect-api-1  | 
prefect-api-1  | View the API reference documentation at <http://0.0.0.0:4200/docs>
prefect-api-1  | 
prefect-api-1  | Check out the dashboard at <http://0.0.0.0:4200>
prefect-api-1  | 
prefect-api-1  | 
prefect-api-1  | 
prefect-api-1  | 22:23:09.266 | DEBUG   | prefect.prefect_redis.messaging - Consumer group already exists: BUSYGROUP Consumer Group name already exists
I can confirm that there is no local process running on port :4200 that would have preventing traffic from getting into my containers mapped port here is the output of my config, grepping for prefect_api-like variables
Copy code
(sourcing) ➜  sourcing git:(SOURC-149--baseline-native-sla-flow) ✗ prefect config view | egrep 'PREFECT_API_URL|PREFECT_API_KEY|PREFECT_UI_URL'
PREFECT_API_URL='<http://localhost:4200/api>' (from .env file)
m
✅ thought for 82.7 seconds
Short answer: yes — those API container logs look normal. You generally won’t see per-connection WebSocket or “events client” logs in the API container. The logs that matter for task visibility live in the process that’s running your flow (worker/process/docker/k8s), not in the API container. Why this matters - In Prefect 3, individual Task Runs in the UI are streamed over a WebSocket from the process executing your flow to the API. - If that client can’t open the WebSocket, your flow still runs and the flow run appears, but task runs won’t. - The API container does not typically log each WS connection by default, so “no websocket logs in the API container” is expected. What to check next (from the environment where the flow actually runs) 1) Turn on debug logs and run again - For a script:
Copy code
PREFECT_LOGGING_LEVEL=DEBUG python your_flow.py
- For a worker or container, set the env var on that process. - In the client logs, look for messages about “events”, “websocket”, or failures to connect. 2) Confirm the API URL inside the runtime that executes your flow - If your flow runs on your host: 
<http://localhost:4200/api>
is correct. - If your flow runs inside another Docker container (worker, docker infra, etc.), 
localhost:4200
points back to that container, not your API. Use the compose service name: - Inside containers, set:
Copy code
PREFECT_API_URL=<http://prefect-api:4200/api>
- Log what the flow sees:
Copy code
from prefect.settings import PREFECT_API_URL
  from prefect import flow, get_run_logger

  @flow
  def debug_settings():
      logger = get_run_logger()
      <http://logger.info|logger.info>(f"PREFECT_API_URL seen by flow: {PREFECT_API_URL.value()}")

  if __name__ == "__main__":
      debug_settings()
3) Run your minimal example with DEBUG logging - You should see either a successful events/websocket start or warnings about failing to connect. If you see failures, it’s almost always a networking/proxy/Docker networking issue. Common pitfalls - Using 
<http://localhost:4200/api>
in a Dockerized runtime. Inside containers, use the compose service name (e.g., `http://prefect-api:4200/api`). - Corporate proxy or reverse proxy that doesn’t forward websockets. - Mismatch between how you “submit” the flow and where the API is reachable (e.g., deployment runs inside a container with different networking than your shell where you ran 
prefect config view
). If you can share: - How you’re submitting the flow (local script vs deployment) - Where the flow executes (host, worker container, Docker/K8s infra) - The DEBUG logs from that runtime (look for “events”/“websocket” lines) …then I can pinpoint the exact fix. Reference: Prefect’s async examples (awaited tasks) use the same asyncio pattern and will show tasks when the client can open the WS connection to the API: Run work concurrently (async).
b
@Marvin I have attached the logs for a flow run called augmented-owl that did not register the tasks. It is the minimal example that I sent earlier. I then added logging around the PREFECT_API_URL and started another flow. Here is the result:
Copy code
Worker 'ProcessWorker abf6f3c3-9a7c-4be6-b0ef-3d45f8d1bec9' submitting flow run '79fcee1d-890a-4b47-a2be-5c5dfa18d46d'
06:02:56 PM
prefect.flow_runs.worker
Opening process...
06:02:56 PM
prefect.flow_runs.runner
Completed submission of flow run '79fcee1d-890a-4b47-a2be-5c5dfa18d46d'
06:02:56 PM
prefect.flow_runs.worker
Running 1 deployment pull step(s)
06:02:57 PM
prefect.flow_runs
 > Running set_working_directory step...
06:02:57 PM
prefect.flow_runs
Changing working directory to '/app'
06:02:57 PM
prefect.flow_runs
Importing flow code from 'sourcing/sla/test_flow.py:test_task_visibility_flow'
06:02:57 PM
prefect.flow_runs
Beginning flow run 'fierce-grebe' for flow 'test-task-visibility-flow'
06:02:57 PM
prefect.flow_runs
Executing flow 'test-task-visibility-flow' for flow run 'fierce-grebe'...
06:02:57 PM
prefect.flow_runs
PREFECT_API_URL: <http://prefect-api:4200/api>
06:02:57 PM
prefect.flow_runs
worker-logs.txt
m
✅ thought for 63.4 seconds
Thanks for the extra detail — that helps. What your logs tell us - The API container logs you shared look normal; it won’t typically log each WebSocket connection. - In the flow runtime, 
PREFECT_API_URL
is correctly set to 
<http://prefect-api:4200/api>
. That’s exactly what you want inside Docker Compose. Next, let’s verify the piece that usually causes missing Task Runs in the UI: the events/WebSocket connection from the process that executes your flow. Please try these quick checks from the environment where the flow runs (the flow subprocess started by the ProcessWorker): 1) Enable DEBUG logs for the flow/worker process - Set this on your worker container (env or docker compose) and re-run:
Copy code
PREFECT_LOGGING_LEVEL=DEBUG
- Re-run your minimal async flow. In the worker/flow logs, look for lines mentioning “events”, “websocket”, or “events client”. If the WebSocket can’t be opened, you’ll usually see a warning/error there. 2) Verify the WebSocket endpoint is reachable from the worker container - Exec into the worker container and probe the WS route (HTTP probe is enough to test reachability):
Copy code
docker compose exec <your-worker-service> sh -lc "wget -S -O - <http://prefect-api:4200/api/ws> || true"
You don’t need a true WS handshake here; we just want to avoid connection refused/timeout. Any HTTP response (even 404/426) proves reachability. 3) Sanity-check with submit() to force TaskRuns Try this small change to your minimal flow to force explicit task runs:
Copy code
@flow(timeout_seconds=60)
async def test_task_visibility_flow(input_value: str = "test") -> str:
    logger = get_run_logger()
    <http://logger.info|logger.info>(f"Flow starting with input: {input_value}")

    f1 = task_one.submit(input_value)
    f2 = task_two.submit(input_value)
    r1 = await f1   # or await f1.result() if you prefer
    r2 = await f2

    f3 = task_three.submit(r1, r2)
    r3 = await f3

    <http://logger.info|logger.info>(f"Flow complete with result: {r3}")
    return r3
- If these 
.submit()
calls produce Task Runs in the UI, but direct 
await task_one(...)
did not, please let me know — that’d narrow it to an async call wrapping issue rather than connectivity. 4) Confirm versions - Inside the worker container:
Copy code
prefect version
- Inside the API container:
Copy code
docker compose exec prefect-api prefect version
- Please share both. Task visibility relies on event streaming introduced in Prefect 3.x; a version mismatch or older server can be the culprit. If the DEBUG logs show any WebSocket or events client warnings/errors, paste those and I’ll zero in on the fix. If the WS probe can’t reach 
<http://prefect-api:4200/api/ws>
, it’s a Docker networking issue. Otherwise, we’ll dig into task invocation vs. engine wrapping.
b
@Marvin 1. here is a snippet from the worker logs. the websocket looks connected
Copy code
prefect-worker-1  | 22:54:03.731 | DEBUG   | prefect.client - Connecting to API at <http://prefect-api:4200/api/>
prefect-worker-1  | 22:54:03.731 | DEBUG   | prefect.client - Connecting to API at <http://prefect-api:4200/api/>
prefect-worker-1  | 22:54:03.731 | DEBUG   | prefect.events.clients - Reconnecting websocket connection.
prefect-worker-1  | 22:54:03.731 | DEBUG   | prefect.events.clients - Opening websocket connection.
prefect-worker-1  | 22:54:03.733 | DEBUG   | prefect.events.clients - Pinging to ensure websocket connected.
prefect-worker-1  | 22:54:03.733 | DEBUG   | prefect.events.clients - Pong received. Websocket connected.
prefect-worker-1  | 22:54:03.733 | DEBUG   | prefect.events.clients - Resending 0 unconfirmed events.
prefect-worker-1  | 22:54:03.733 | DEBUG   | prefect.events.clients - Finished resending unconfirmed events.
prefect-worker-1  | 22:54:03.739 | DEBUG   | prefect.workers.process.processworker abf6f3c3-9a7c-4be6-b0ef-3d45f8d1bec9 - Worker synchronized with the Prefect API server.
prefect-worker-1  | Worker 'ProcessWorker abf6f3c3-9a7c-4be6-b0ef-3d45f8d1bec9' started!
2. the websocket endpoint is reachable and returns a 401
Copy code
(sourcing) ➜  sourcing git:(SOURC-149--baseline-native-sla-flow) ✗ docker compose exec prefect-worker sh -lc "wget -S -O - <http://prefect-api:4200/api/ws> || true"
Connecting to prefect-api:4200 (172.18.0.4:4200)
  HTTP/1.1 401 Unauthorized
wget: server returned error: HTTP/1.1 401 Unauthorized
3. I updated the flow to use submit as follows, and still did not get any tasks to populate in the UI
Copy code
@flow(timeout_seconds=60)
async def test_task_visibility_flow(input_value: str = "test") -> str:
    logger = get_run_logger()
    <http://logger.info|logger.info>(f"PREFECT_API_URL: {PREFECT_API_URL}")
    <http://logger.info|logger.info>(f"Flow starting with input: {input_value}")
    <http://logger.info|logger.info>("Running with submit instead of await...")

    r1 = task_one.submit(input_value).result()
    r2 = task_two.submit(input_value).result()
    r3 = task_three.submit(r1, r2).result()

    <http://logger.info|logger.info>(f"Flow complete with result: {r3}")
    return r3
4. the prefect command does not seem to be available in either the worker or api container, but I can confirm that we are using 3.6.5
Copy code
(sourcing) ➜  sourcing git:(SOURC-149--baseline-native-sla-flow) ✗ docker compose exec prefect-api prefect version
OCI runtime exec failed: exec failed: unable to start container process: exec: "prefect": executable file not found in $PATH
(sourcing) ➜  sourcing git:(SOURC-149--baseline-native-sla-flow) ✗ docker compose exec prefect-worker prefect version
OCI runtime exec failed: exec failed: unable to start container process: exec: "prefect": executable file not found in $PATH
(sourcing) ➜  sourcing git:(SOURC-149--baseline-native-sla-flow) ✗
m
✅ thought for 208.3 seconds
Thanks — this is very helpful. What we know so far - The worker can reach the API and the events WebSocket connects successfully (Pong received). That rules out the most common client-side cause. - Hitting /api/ws returns 401 from inside the worker — that’s expected for an HTTP GET; it still proves the route is reachable. - Switching to 
.submit()
didn’t surface Task Runs, which suggests the issue is not how tasks are invoked. Given that, the most likely cause is on the server side: the Task Run Recorder service is not running. Why this matters - In Prefect 3, Task Runs in the UI are materialized by a background service in the server called “TaskRunRecorder.” The flow/process emits task-run events over the WebSocket. The server consumes those events and writes TaskRun rows to the DB. - If the TaskRunRecorder service is disabled or not starting, flows will still run and appear, but individual Task Runs won’t be recorded — exactly what you’re seeing. Action items to verify on the API container 1) Enable DEBUG logging on the API container - Set 
PREFECT_LOGGING_LEVEL=DEBUG
in the API container and restart it. - Then grep logs for the recorder: - “TaskRunRecorder” or “services.task_run_recorder” - Errors mentioning “events subscriber”, “redis/messaging”, or “failed to process event” 2) Ensure the recorder service is enabled - Explicitly set this env var on the API container:
Copy code
PREFECT_SERVER_SERVICES_TASK_RUN_RECORDER_ENABLED=true
- Then restart the API container and watch logs for it starting. 3) Confirm the message broker is reachable from the API container - Your API logs show:
Copy code
prefect.prefect_redis.messaging - Consumer group already exists
That’s a good sign. Still, if you have Redis in your compose, make sure the API container can reach it at the correct hostname/port (usually the compose service name). 4) Note the “Using profile 'ephemeral'” line - Your API logs showed:
Copy code
Using profile 'ephemeral'
- That profile may not have the server services enabled by default. Consider setting 
PREFECT_PROFILE=default
(or your custom profile) in the API container, or just explicitly set the service env var above to force-enable the TaskRunRecorder. 5) Verify server/worker image versions - You mentioned 3.6.5 — great. Please ensure both the API and worker images are actually running 3.6.5. A mismatch with an older server could prevent task events from being understood/recorded. - If you can’t run 
prefect version
in the containers, please share the image tags for 
prefect-api
and 
prefect-worker
from your compose file. Optional quick checks - After a flow run, query the server for task-run counts:
Copy code
curl -s -X POST <http://localhost:4200/api/ui/task_runs/dashboard/counts> -H 'Content-Type: application/json' -d '{}'
If this returns zeros while you know tasks ran, it corroborates that nothing is recording task runs. - Programmatic check from your host using the same API URL:
Copy code
from prefect import get_client
  import asyncio

  async def main():
      async with get_client() as c:
          trs = await c.read_task_runs(limit=5)
          print(len(trs), trs[:1])

  asyncio.run(main())
If zero, again points to recorder not persisting anything. Why I’m confident this is server-side - Client shows successful WebSocket connection. - 
.submit()
didn’t change behavior. - This symptom pattern (flows visible, tasks not) is what we see when the TaskRunRecorder isn’t running. Reference - TaskRunRecorder service (consumes task-run events and writes TaskRuns): TaskRunRecorder source If you can share: - The docker-compose service definition for 
prefect-api
(env vars especially) - Any API container DEBUG logs around startup mentioning “TaskRunRecorder” or errors I can point out the exact config tweak.
b
@Marvin here is the docker compose file
Copy code
x-api-template: &api-base
  build:
    context: .
    dockerfile: Dockerfile
  depends_on:
    database:
      condition: service_healthy
    redis:
      condition: service_started
    migrate:
      condition: service_completed_successfully
  restart: unless-stopped
  develop:
    watch:
      - action: sync+restart
        path: .env
        target: /app/.env
      - action: rebuild
        path: ./Dockerfile
      - action: rebuild
        path: ./pyproject.toml
        target: /app/pyproject.toml
      - action: rebuild
        path: ./poetry.lock
        target: /app/poetry.lock
      - action: sync+restart
        path: ./sourcing/scripts/
        target: /app/scripts/
      - action: sync+restart
        path: ./sourcing/
        target: /app/sourcing/
  env_file:
    - .env
  volumes:
    - ./sourcing:/app/sourcing:ro
    - ~/.aws/:/home/appuser/.aws:ro

x-prefect-env-variables: &prefect-env
  PREFECT_API_URL: "<http://prefect-api:4200/api>"
  PREFECT_SERVER_API_PORT: 4200
  PREFECT_SERVER_API_HOST: "0.0.0.0"
  PREFECT_SERVER_ANALYTICS_ENABLED: false
  PREFECT_SERVER_EVENTS_CAUSAL_ORDERING: prefect_redis.ordering
  PREFECT_SERVER_CONCURRENCY_LEASE_STORAGE: prefect_redis.lease_storage
  PREFECT_SERVER_API_AUTH_STRING: ${PREFECT_SERVER_API_AUTH_STRING}
  PREFECT_API_AUTH_STRING: ${PREFECT_API_AUTH_STRING}
  PREFECT_API_DATABASE_CONNECTION_URL: "<postgresql+asyncpg://username:password@database:5432/prefect_db>"
  PREFECT_API_DATABASE_MIGRATE_ON_START: false
  PREFECT_API_DATABASE_TIMEOUT: 600
  PREFECT_REDIS_MESSAGING_HOST: redis
  PREFECT_REDIS_MESSAGING_PORT: 6379
  PREFECT_REDIS_MESSAGING_DB: 15
  PREFECT_MESSAGING_BROKER: prefect_redis.messaging
  PREFECT_MESSAGING_CACHE: prefect_redis.messaging
  PREFECT_LOGGING_LEVEL: DEBUG
  CACHE__REDIS_URL: redis://:@redis:6379?db=14

services:
  database:
    image: pgvector/pgvector:pg15
    user: root
    volumes:
      - pg_database:/var/lib/postgresql
    environment:
      POSTGRES_USER: username
      POSTGRES_PASSWORD: password
      POSTGRES_DB: prefect_db
      POSTGRES_HOST_AUTH_METHOD: trust
    restart: unless-stopped
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U username -d prefect_db"]
      interval: 5s
      timeout: 60s
      retries: 30
    command: ["postgres", "-c", "log_statement=all", "-c", "log_destination=stderr"]

  redis:
    image: redis:7.4-alpine
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "redis-cli", "--raw", "incr", "ping"]

  migrate:
    <<: *api-base
    restart: on-failure
    depends_on:
      database:
        condition: service_healthy
    command: ./scripts/migrate.sh
    environment:
      <<: *prefect-env

  api:
    <<: *api-base
    command: ./scripts/api-dev.sh
    healthcheck:
      test: ["CMD-SHELL", "curl --fail <http://0.0.0.0:8001/health>"]
      interval: 5s
      timeout: 10s
      retries: 30
    ports:
    - 8001:8001
    deploy:
      replicas: 1
    environment:
      <<: *prefect-env

  prefect-api:
    <<: *api-base
    command: ./scripts/prefect-api.sh
    ports:
    - 4200:4200
    deploy:
      replicas: 1
    healthcheck:
      test: ["CMD-SHELL", "curl --fail <http://0.0.0.0:4200/api/health>"]
      interval: 5s
      timeout: 10s
      retries: 30
    environment:
      <<: *prefect-env
      PREFECT_API_URL: "<http://localhost:4200/api>"

  prefect-worker:
    <<: *api-base
    command: ./scripts/prefect-worker.sh --pool default-agent-pool --limit 1
    healthcheck:
      test: ["CMD-SHELL", "curl --fail <http://0.0.0.0:8080/health>"]
      interval: 5s
      timeout: 10s
      retries: 5
      start_period: 30s
    deploy:
      replicas: 1
    environment:
      <<: *prefect-env
    depends_on:
      prefect-api:
        condition: service_healthy
      database:
        condition: service_healthy
      redis:
        condition: service_started

volumes:
  pg_database:
m
✅ thought for 204.7 seconds
This is great signal — thanks for sharing the compose and the worker logs. What your logs + compose tell me - The worker’s WebSocket is connecting and healthy (Pong received). That rules out client-side WS issues. - Your compose is running a custom server command (
./scripts/prefect-api.sh
) and the API container prints “Using profile 'ephemeral'”. That’s a strong hint that the server is not starting its background services. - In Prefect 3, Task Runs in the UI are written by a background service in the server called TaskRunRecorder. If that service is not running, flows will show up but tasks will not — exactly what you’re seeing. Two concrete things to fix on the server side 1) Make sure the TaskRunRecorder service is enabled and started - Explicitly enable it on the API container:
Copy code
PREFECT_SERVER_SERVICES_TASK_RUN_RECORDER_ENABLED=true
- Ensure your server command starts services. The simplest is to run the canonical entrypoint:
Copy code
prefect server start
Avoid running in “no services” mode or any custom invocation that only runs the webserver. If your 
./scripts/prefect-api.sh
wraps a custom gunicorn/uvicorn command, those do not start Prefect services by default. 2) Prefer the server-scoped messaging settings - You already point messaging at Redis, which is right. To be explicit for Prefect 3 server components, set:
Copy code
PREFECT_SERVER_EVENTS_MESSAGING_BROKER=prefect_redis.messaging
  PREFECT_SERVER_EVENTS_MESSAGING_CACHE=prefect_redis.messaging
- You can keep the 
PREFECT_REDIS_MESSAGING_*
connection settings you have — those are used by the Redis integration. Suggested changes to your compose (only the Prefect bits shown)
Copy code
x-prefect-env-variables: &prefect-env
  PREFECT_API_URL: "<http://prefect-api:4200/api>"
  PREFECT_SERVER_API_PORT: 4200
  PREFECT_SERVER_API_HOST: "0.0.0.0"
  PREFECT_SERVER_ANALYTICS_ENABLED: false

  # Messaging via Redis for events
  PREFECT_SERVER_EVENTS_MESSAGING_BROKER: prefect_redis.messaging
  PREFECT_SERVER_EVENTS_MESSAGING_CACHE: prefect_redis.messaging
  PREFECT_REDIS_MESSAGING_HOST: redis
  PREFECT_REDIS_MESSAGING_PORT: 6379
  PREFECT_REDIS_MESSAGING_DB: 15

  # Enable TaskRunRecorder explicitly
  PREFECT_SERVER_SERVICES_TASK_RUN_RECORDER_ENABLED: "true"

  PREFECT_LOGGING_LEVEL: DEBUG

services:
  prefect-api:
    <<: *api-base
    # Prefer the stock server entrypoint to ensure services are started
    command: sh -lc "prefect server start"
    ports:
      - 4200:4200
    environment:
      <<: *prefect-env
      # Keep API URL inside the API container as localhost OR drop this override entirely
      PREFECT_API_URL: "<http://localhost:4200/api>"
Three quick validations after you apply/restart - API container logs (DEBUG) should mention the TaskRunRecorder starting. If you see errors around events subscriber/messaging, share those. - From the worker container, your existing WebSocket handshake “Pong received” should still appear. - Re-run your minimal flow. Tasks should now materialize in the UI. Extra notes - The “Using profile 'ephemeral'” line in your API logs suggests the server is starting with a profile that may not have services configured. You can set 
PREFECT_PROFILE=default
on the API container or rely on explicit env vars (as above) to force-enable the recorder. - 
.submit()
vs. 
await
won’t matter if the recorder isn’t running; your test was useful and supports the server-side hypothesis. - The 401 from 
wget <http://prefect-api:4200/api/ws>
is expected; it’s just a basic HTTP GET without auth. Your worker log proves the authenticated WS connection is fine. References - TaskRunRecorder service (the component that writes Task Runs from events): TaskRunRecorder source - Server services settings (env var names): Server services settings
- Events/messaging settings: Server events settings If you can share the contents of 
./scripts/prefect-api.sh
, I can confirm whether it’s skipping services and give an exact command replacement.
b
@Marvin the 
--no-services
flag was in the script. this is now resolved
m
✅ thought for 17.0 seconds
Awesome — that explains it. The 
--no-services
flag disables all server background services, including the TaskRunRecorder that writes Task Runs to the DB. With services enabled, your tasks should now populate in the UI. Quick verifications: - Trigger a new run of your minimal flow; you should see Task Runs appear under the flow run. - In API logs (DEBUG), you should see the TaskRunRecorder start up. If you want to be explicit:
Copy code
PREFECT_SERVER_SERVICES_TASK_RUN_RECORDER_ENABLED=true
- Note: past runs won’t be backfilled; only new runs will have task visibility. Optional sanity checks: - Keep your Redis/event messaging settings as you have them — they’re appropriate for Prefect 3 Server. - Either 
await task(...)
or 
task.submit(...).result()
will surface Task Runs now that the recorder is running. References: - TaskRunRecorder service (writes TaskRuns) - Server services settings (env vars) If anything still looks off, share the API container startup logs after the change and I’ll help double-check.
b
@Marvin what else does 
--no-services
disable, and what side affects might that have?
m
✅ thought for 87.2 seconds
Short version - 
prefect server start
runs the API/UI plus a set of background “services” that do orchestration, events, automations, and cleanup. - 
--no-services
turns off those background services. You’ll still have the API/UI, but most orchestration features stop. What 
--no-services
disables and the effects - Scheduler - Effect: Deployment schedules won’t create new flow runs automatically. You must trigger runs manually. - Mark late runs - Effect: Flow runs that miss their scheduled start won’t be marked Late. - Foreman (work pool/worker health) - Effect: Worker/work pool health/heartbeats aren’t reconciled; stale/offline status won’t be updated. - Repossessor (concurrency lease cleanup) - Effect: Expired concurrency leases won’t be reclaimed; concurrency slots can remain stuck. - TaskRunRecorder - Effect: Task run state changes (emitted as events) won’t be written to the database → tasks won’t appear in the UI even though flows run. - Event persister - Effect: Events from the message bus aren’t persisted; event-driven features lose history/state. - Automations: reactive and proactive triggers - Effect: Automations won’t evaluate (no event-driven or scheduled triggers), and their actions won’t run. - Actions executor - Effect: Automation actions (notifications, run deployment, etc.) won’t execute. - Stream distributor (event streaming to subscribers) - Effect: Live event streams for dashboards/subscribers are disabled; some real-time UI updates won’t occur. - Pause expiration handler - Effect: Paused flow runs won’t auto-fail/timeout when their pause window expires. - Cancellation cleanup - Effect: Cancelling a flow may not cascade cancellation to child tasks/subflows automatically. What still runs with 
--no-services
- API and UI webserver - Telemetry/analytics (it’s allowed to run in webserver-only mode unless you disable it with 
PREFECT_SERVER_ANALYTICS_ENABLED=false
) How to control services explicitly - You can toggle individual services via env vars, e.g.: - 
PREFECT_SERVER_SERVICES_TASK_RUN_RECORDER_ENABLED=true
- 
PREFECT_SERVER_SERVICES_SCHEDULER_ENABLED=true
- 
PREFECT_SERVER_SERVICES_EVENT_PERSISTER_ENABLED=true
- and corresponding 
_LOOP_SECONDS
, 
_BATCH_SIZE
, etc. for tuning - For Redis-backed events/messaging, prefer the server-scoped settings: - 
PREFECT_SERVER_EVENTS_MESSAGING_BROKER=prefect_redis.messaging
- 
PREFECT_SERVER_EVENTS_MESSAGING_CACHE=prefect_redis.messaging
When to use 
--no-services
- Only if you’re deliberately running services separately (e.g., scaling webservers independently and using 
prefect server services start
elsewhere). - Otherwise, avoid it — it will lead to exactly the symptoms you saw (no task runs in UI, automations not firing, schedules not creating runs, etc.). If you want, paste the startup logs after removing 
--no-services
and I can confirm each service started successfully.
b
@Marvin would flow timeouts still work correctly with 
--no-services
? for example 
@flow(_timeout_seconds_=600)
m
✅ thought for 45.2 seconds
Yes. Flow timeouts still work with 
--no-services
. Details: - The 
timeout_seconds
on 
@flow
is enforced by the flow engine in the process that’s executing your flow (e.g., the worker’s subprocess). It does not depend on server background services. When the timeout elapses, the engine will end the run and set the flow state accordingly (TimedOut/Failed) via the API. - You should still see the flow run transition to a terminal timeout/failed state in the UI (assuming the API is reachable), even if services are disabled. What you won’t get with `--no-services`: - Task Runs in the UI (TaskRunRecorder is disabled, so task states aren’t written to the DB). - Automations reacting to the timeout (event persister and trigger evaluators are disabled). - Scheduler, Late-run marking, pause expirations, cancellation cleanup, and live event streaming. Notes: - Task-level 
timeout_seconds
are also enforced locally by the engine, but without TaskRunRecorder you won’t see those task states in the UI. - Make sure you’re using 
timeout_seconds
(not 
_timeout_seconds_
) on the decorator:
Copy code
@flow(timeout_seconds=600)
  async def my_flow():
      ...
👍 1
17 Views
Open in Slack
PreviousNext
Powered by