https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> I got this when start prefect worker,...
# ask-marvin
t
@Marvin I got this when start prefect worker, is there anyway to increase the pool timeout?
Copy code
06:51:10.060 | DEBUG   | prefect.client - Encountered retryable exception during request. Another attempt will be made in 2.1943795936861705s. This is attempt 1/6.
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions
    yield
  File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 394, in handle_async_request
    resp = await self._pool.handle_async_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 256, in handle_async_request
    raise exc from None
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 232, in handle_async_request
    connection = await pool_request.wait_for_connection(timeout=timeout)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 35, in wait_for_connection
    await self._connection_acquired.wait(timeout=timeout)
  File "/usr/local/lib/python3.12/site-packages/httpcore/_synchronization.py", line 149, in wait
    with map_exceptions(anyio_exc_map):
  File "/usr/local/lib/python3.12/contextlib.py", line 155, in __exit__
    self.gen.throw(value)
  File "/usr/local/lib/python3.12/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.PoolTimeout

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/prefect/client/base.py", line 268, in _send_with_retry
    response = await send(request, *send_args, **send_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1629, in send
    response = await self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1657, in _send_handling_auth
    response = await self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1694, in _send_handling_redirects
    response = await self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1730, in _send_single_request
    response = await transport.handle_async_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 393, in handle_async_request
    with map_httpcore_exceptions():
  File "/usr/local/lib/python3.12/contextlib.py", line 155, in __exit__
    self.gen.throw(value)
  File "/usr/local/lib/python3.12/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.PoolTimeout
m
✅ thought for 192.0 seconds
Short answer: yes. That 
httpx.PoolTimeout
is coming from the HTTP connection pool used by the Prefect client. In Prefect 3.x you can increase the client’s timeouts (including the pool timeout) via an environment variable, and optionally increase client retries. What to set - Increase all HTTPX timeouts (connect/read/write/pool) used by the Prefect client: - 
PREFECT_API_REQUEST_TIMEOUT
(seconds) - Optionally increase retries for retryable errors (like 
PoolTimeout
): - 
PREFECT_CLIENT_MAX_RETRIES
- 
PREFECT_CLIENT_RETRY_JITTER_FACTOR
(optional) - 
PREFECT_CLIENT_RETRY_EXTRA_CODES
(optional) Examples - Shell (before starting the worker):
Copy code
export PREFECT_API_REQUEST_TIMEOUT=120
export PREFECT_CLIENT_MAX_RETRIES=10
prefect worker start -p <your-pool>
- Docker Compose:
Copy code
services:
  prefect-worker:
    image: prefecthq/prefect:3-latest
    command: prefect worker start -p your-pool
    environment:
      PREFECT_API_URL: <https://api.prefect.cloud/api/accounts/<acct>/workspaces/<ws>>
      PREFECT_API_KEY: <your-key>
      PREFECT_API_REQUEST_TIMEOUT: "120"
      PREFECT_CLIENT_MAX_RETRIES: "10"
- Kubernetes (snippet):
Copy code
env:
  - name: PREFECT_API_REQUEST_TIMEOUT
    value: "120"
  - name: PREFECT_CLIENT_MAX_RETRIES
    value: "10"
Notes - Workers use the standard Prefect client under the hood, so they’ll pick these up from the environment automatically; there isn’t a dedicated CLI flag just for the pool timeout. - If you continue to see pool timeouts, you may be saturating connections (e.g., due to proxy/network constraints). Prefect’s client also limits concurrent connections by default for stability; this isn’t currently adjustable via environment variables. If you need to tune HTTPX limits at a low level, that requires creating a custom client with 
httpx_settings
in code, which the CLI worker does not expose. If you can share a bit more about your setup (Prefect Cloud vs self-hosted API, where the worker runs, presence of proxies), I can suggest more targeted tweaks.
t
@Marvin what is the default value of PREFECT_API_REQUEST_TIMEOUT
m
✅ thought for 56.0 seconds
60.0 seconds by default. Details: - It’s defined in Prefect 3.x settings (
APISettings.request_timeout
) and maps to the env var 
PREFECT_API_REQUEST_TIMEOUT
. - Programmatic check:
Copy code
from prefect.settings import PREFECT_API_REQUEST_TIMEOUT
PREFECT_API_REQUEST_TIMEOUT.value()  # -> 60.0
19 Views
Open in Slack
PreviousNext
Powered by