Janet Carson
05/29/2026, 9:06 PMMarvin
05/29/2026, 9:06 PMMarvin
05/29/2026, 9:08 PM--work-queue is not required.
I verified the CLI help for prefect worker start, and only --pool is required. --work-queue is optional and defaults to polling all queues in the work pool.
So this is valid:
bash
prefect worker start --pool my-pool
And --work-queue is only needed if you want to restrict the worker to specific queues:
bash
prefect worker start --pool my-pool --work-queue queue-a --work-queue queue-b
I also checked for recent changes and didn’t find anything indicating that this became required in the last day.
Relevant docs:
- Workers documentation
If you’re seeing an error that implies --work-queue is required, paste the exact command and error text here and I can help pinpoint what’s actually going on.Janet Carson
05/29/2026, 10:12 PMMarvin
05/29/2026, 10:12 PMMarvin
05/29/2026, 10:14 PMbash
prefect worker start --pool my-pool
then it should poll all queues in that pool, and those queues should typically become Ready after polling.
So if the pool is Ready but the queues are Not Ready, likely explanations are:
1. The worker is online, but not actually polling those queues successfully
- heartbeat is reaching the API
- queue polling may be failing or not happening
2. A transient UI/backend timing issue
- statuses are updated asynchronously
- you may briefly see inconsistent state between pool and queues
3. The worker is connected to the pool, but filtering/config means it is not polling the queues you expect
- less likely if you truly started it with only --pool
4. The queues aged out of readiness
- queue readiness is time-based
- if a queue hasn’t been polled recently, it flips back to Not Ready
About the conflicting UI text:
- The queue page title/header saying “ready to go” sounds like more of a generic UI message
- The sidebar status is more likely reflecting the actual stored queue readiness state
So if those disagree, I’d trust the explicit Status: Not Ready more than the page title copy.
What I’d check next:
1. Worker logs
Look for repeated polling activity or errors.
You want to see signs it is:
- connected to the correct pool
- requesting scheduled flow runs
- not erroring on API calls
2. Pool/queue names
Confirm the worker is attached to the exact pool you’re viewing in the UI.
3. Whether runs are actually being submitted to those queues
If deployments target a different pool or queue, the worker may be healthy but irrelevant to those queues.
4. Refresh after ~1 minute
Since queue readiness is tied to recent polling, it should usually settle quickly if polling is working.
If you want, paste:
- your worker start command
- a snippet of worker logs from startup and a few polling cycles
- whether you’re on Cloud or Server / self-hosted
and I can help narrow down why the queues are staying Not Ready.Janet Carson
05/29/2026, 10:19 PMMarvin
05/29/2026, 10:19 PMMarvin
05/29/2026, 10:20 PMREADY if it has been polled by a worker within the last 60 seconds. If it goes longer than that without a poll, it will age back to NOT_READY.
Docs:
- Work pools and work queues
So practically:
- polled within ~1 minute → READY
- no poll for > ~1 minute → NOT_READY
If you want, I can also help you map that to the worker polling cadence you should expect in logs.Janet Carson
05/29/2026, 10:20 PMMarvin
05/29/2026, 10:20 PMMarvin
05/29/2026, 10:22 PMbash
PREFECT_SERVER_SERVICES_FOREMAN_WORK_QUEUE_LAST_POLLED_TIMEOUT_SECONDS
There’s also a legacy alias:
bash
PREFECT_API_SERVICES_FOREMAN_WORK_QUEUE_LAST_POLLED_TIMEOUT_SECONDS
And the worker polling interval is separately configurable with:
bash
PREFECT_WORKER_QUERY_SECONDS
Defaults are roughly:
- PREFECT_WORKER_QUERY_SECONDS=10
- PREFECT_SERVER_SERVICES_FOREMAN_WORK_QUEUE_LAST_POLLED_TIMEOUT_SECONDS=60
So the relationship is:
- worker polls every ~10s by default
- queue is marked Not Ready if it hasn’t been polled for ~60s
If you’re self-hosting, you can tune both. If you’re using Prefect Cloud, server-side foreman settings generally aren’t user-configurable.
Relevant docs:
- Work pools and work queues
- Settings reference
If you want, I can suggest sane values depending on whether your workers are long-lived, bursty, or running in Kubernetes.Janet Carson
05/29/2026, 10:28 PMMarvin
05/29/2026, 10:42 PMMarvin
05/29/2026, 10:43 PMtext
received 1013 (try again later) service_unavailable
means the WebSocket was closed with code 1013, which generally means:
- service unavailable
- temporary overload
- try again later
So this usually points to a temporary API-side or network-side disruption, not necessarily a fatal worker problem.
What this log implies:
- good news: the worker detected the channel problem and handled it correctly
- good news: fallback is active, so the worker can often continue operating
- possible concern: the API, proxy, load balancer, or network path briefly refused or interrupted the WebSocket connection
When to worry:
- If you see it occasionally: usually not a big deal
- If you see it constantly or continuously: then something is persistently breaking WebSocket connectivity
Common causes:
- temporary Prefect API unavailability
- reverse proxy / ingress timeouts on WebSockets
- load balancer behavior
- transient network instability
- server under load
If your flows are still being picked up and executed, this log by itself is usually just informational/debug noise indicating degraded-but-functional behavior.
If runs are not being picked up, then I’d investigate:
1. worker logs around the same timestamp
2. API / server health
3. any ingress or proxy in front of Prefect
4. whether the message repeats every few seconds/minutes
Docs:
- Workers
If you want, paste a larger log snippet around that message and I can help tell whether it’s harmless fallback behavior or a real connectivity problem.Janet Carson
05/29/2026, 10:47 PMMarvin
05/29/2026, 10:47 PMMarvin
05/29/2026, 10:47 PM1013 ... service_unavailable
What to ask IT to check
Tell them your worker needs outbound connectivity from the worker host/container to your Prefect API endpoint over HTTPS / WSS.
Specifically, ask them to check for:
- blocked outbound WebSocket connections to the Prefect API hostname
- TLS inspection / SSL interception that may interfere with upgraded WebSocket connections
- proxy/load balancer timeouts on long-lived connections
- HTTP Upgrade header stripping or rejection
- idle connection termination
- rate limiting / service unavailable responses coming from a proxy or gateway
- destination hostname or port filtering for the Prefect API URL
Useful wording for IT
You can give them something like:
text
We have a service that needs outbound HTTPS and secure WebSocket (WSS) connectivity to <prefect-api-hostname>:443.
Please check whether any firewall, proxy, SSL inspection appliance, or ingress layer is:
- blocking or resetting WebSocket connections
- rejecting HTTP Upgrade requests
- closing long-lived idle connections
- returning 1013 / service unavailable responses
- performing TLS interception that breaks WSS
What should work
Your worker should be able to connect to:
- https://... for normal REST API traffic
- wss://... for the worker channel
If REST works but WSS does not, that strongly suggests:
- firewall/proxy rules
- ingress config
- or SSL interception issues
What evidence IT should look for in logs
Ask them to search logs for the worker machine’s source IP and the Prefect API destination, especially around the timestamps where you see the debug message.
Helpful indicators:
- denied or dropped outbound CONNECT or Upgrade: websocket requests
- proxy responses like 403, 502, 503, 504
- connection resets / TCP resets
- TLS handshake failures
- certificate validation or certificate substitution events
- WebSocket upgrade failures
- long-lived connection timeout or idle timeout events
- service-unavailable responses generated by a proxy/load balancer rather than the Prefect app itself
Headers / protocol details they may care about
If they inspect HTTP logs, they should look for WebSocket upgrade attempts like:
- Connection: Upgrade
- Upgrade: websocket
- requests over HTTPS to your Prefect API host
- successful 101 Switching Protocols versus failure responses
If they never see a successful WebSocket upgrade, that’s a clue.
A few practical checks you can do yourself
From the worker environment, test:
1. basic HTTPS reachability to the API
2. whether a proxy is configured via HTTP_PROXY, HTTPS_PROXY, NO_PROXY
3. whether the worker sits behind:
- corporate proxy
- Kubernetes egress proxy
- ingress/load balancer with short idle timeout
Important nuance
If you are self-hosting Prefect Server, the problem may also be on your own ingress/reverse proxy in front of the API, not just the corporate firewall. In that case, check:
- nginx / envoy / ALB / ingress controller WebSocket support
- idle timeout values
- upgrade header forwarding
- 503 generation upstream
If you want, I can help you build a short network checklist for IT based on whether your worker is:
- on a VM
- in Docker
- in Kubernetes
- behind a corporate proxy
- connecting to Prefect Cloud vs self-hosted Prefect ServerJanet Carson
05/29/2026, 10:48 PMMarvin
05/29/2026, 10:48 PMMarvin
05/29/2026, 10:49 PM503 / service unavailable
- rate-limit or close upgraded connections
Good message to send IT
text
We run Prefect workers inside our network that make outbound connections to Prefect Cloud.
The workers need stable outbound HTTPS and secure WebSocket (WSS) connectivity over port 443 to Prefect Cloud.
Please check whether any firewall, proxy, SSL inspection appliance, or egress control is:
- blocking outbound WSS / WebSocket connections
- rejecting HTTP Upgrade requests
- intercepting TLS and replacing certificates
- closing long-lived connections due to idle timeout
- returning 503/service unavailable responses
- resetting connections after upgrade
What they should look for in logs
Have them search egress logs around the exact timestamps of your worker debug messages for the worker node/IP and Prefect Cloud destination.
Useful things to look for:
- outbound denies to Prefect Cloud hostname on port 443
- proxy errors on CONNECT
- WebSocket upgrade failures
- 403, 429, 502, 503, 504
- TCP reset / connection terminated events
- TLS inspection/certificate substitution events
- idle session timeout events
- policy engine events for “unknown application”, “websocket”, “tunneling”, or “upgrade”
Important nuance for Prefect Cloud
Because this is outbound to a SaaS service, the most likely culprits are:
- corporate web proxy
- SSL inspection appliance
- egress firewall
- CASB / secure web gateway
This is less likely to be an ingress/reverse-proxy misconfiguration on your side, unless your workers must traverse an internal outbound proxy first.
What you can tell if REST fallback is working
If the worker still:
- heartbeats
- stays visible in Cloud
- sometimes picks up runs
then basic HTTPS is probably okay.
If the WebSocket worker channel keeps failing, that usually means:
- WSS specifically is degraded or blocked
- or long-lived connections are being interrupted
What I’d ask you to check locally too
From the worker environment:
- whether HTTPS_PROXY, HTTP_PROXY, or NO_PROXY are set
- whether all outbound traffic must go through a corporate proxy
- whether certificate trust is customized
- whether disconnects happen on a regular cadence, like every 30s / 60s / 5m, which often points to enforced timeout policy
Prefect Cloud links:
- Prefect Cloud
- Workers
If you want, I can help you turn this into a very short IT ticket template plus a self-check command list for your worker host/container.