Sebastian S
12/03/2025, 4:51 PMMarvin
12/03/2025, 4:51 PMMarvin
12/03/2025, 4:54 PMpropose_state(...)
- The client POSTs to the API:
# Conceptual shape in PrefectClient
await self.request(
"POST",
f"/task_runs/{task_run_id}/set_state",
json={
"state": state_create.model_dump(mode="json", serialize_as_any=True),
"force": force,
},
)
- Server applies orchestration, writes state to the DB, returns an OrchestrationResult.
Server endpoints (where the API receives the updates):
- Task run state: https://github.com/PrefectHQ/prefect/blob/main/src/prefect/server/api/task_runs.py
- Flow run state: https://github.com/PrefectHQ/prefect/blob/main/src/prefect/server/api/flow_runs.py
Related schema
- State payload (what’s serialized to JSON): https://github.com/PrefectHQ/prefect/blob/main/src/prefect/server/schemas/actions.py (see StateCreate)
Redis in Prefect 3.x
- Redis is not used to transport state updates. The only Redis code in core is the optional block:
- Redis block: https://github.com/PrefectHQ/prefect/blob/main/src/prefect/blocks/redis.py
Notes
- In Prefect 3.x, workers execute flow runs and the flow engine inside that run submits both flow and task run states via the client. There’s no queue/broker-based (e.g., Redis) path for task state updates.
- If you’re actually on Prefect 2.x or looking for logging/heartbeats, let me know and I can point you to those code paths too.Nate
12/03/2025, 4:56 PMMarvin
12/03/2025, 4:56 PMMarvin
12/03/2025, 5:05 PMemit_event(...), which acquires the singleton EventsWorker instance and enqueues the event.
- Queue service base: prefect/_internal/concurrency/services.py
- Provides the queue + background loop pattern that EventsWorker uses (enqueue in the foreground, handle/send in the background).
WebSocket client and send path
- Events clients (WebSocket sender): prefect/events/clients.py
- Builds the WebSocket client and defines the emit(...) call that pushes events to the server.
- WebSocket utilities used by the client: prefect/_internal/websockets.py
- Handles connection details, auth headers, ping/pong, and reconnect/backoff.
Typical sending flow inside EventsWorker (conceptual shape):
# enqueue (synchronous)
EventsWorker.instance().send(event_dict)
# background loop (async)
while True:
event = await queue.get()
await events_client.emit(event) # sends JSON over WS to the API
Where engine/worker emits events for task/flow states
- Task engine emits task-run state change events: prefect/task_engine.py
- Look for utilities like emit_task_run_state_change_event(...).
- Flow/task state proposal utilities (call emit on state changes): prefect/utilities/engine.py
- State transitions here often trigger the corresponding event emission helpers.
Server-side receiver
- Event ingestion endpoint: prefect/server/api/events.py
- Receives events (WebSocket) and forwards them into the server’s event pipeline.
- Event processing pipeline: prefect/server/events/pipeline.py
Relation to task/run state updates
- State updates (accept/reject/wait/abort) still go over HTTP via the orchestration client:
- Client: prefect/client/orchestration/__init__.py
- Engine proposal: prefect/utilities/engine.py
- Events are emitted in parallel via WebSocket by the EventsWorker for telemetry like state-change events. No Redis is involved in this transport.
If you want, I can point you to the specific methods that build the event payload (e.g., event, resource, related, payload) for task-run state changes.Sebastian S
12/03/2025, 5:30 PMevents_worker => (ws) => API server => redis event queue (events pipeline) => (redis) => task_run_recorder => DBNate
12/03/2025, 5:42 PMNate
12/03/2025, 5:42 PMevents/in ws endpoint, which offloads to redis and then some background services consume the redis streamsSebastian S
12/03/2025, 6:39 PM