That error means your SQLAlchemy QueuePool is exhausted: the default pool_size=5 plus max_overflow=10 allows up to 15 concurrent checkouts; a 16th waits up to 30s then times out. You likely have more concurrent DB operations than the pool can satisfy, or connections aren’t being returned. Quick fixes you can try right now - Increase the pool and/or overflow to match peak concurrency - Reduce/constrain concurrency from Prefect so you never exceed the pool - Ensure every connection is returned to the pool (use context managers) and avoid creating a new Engine per task If you’re using Prefect 3.x with prefect-sqlalchemy - Install:

pip install "prefect[sqlalchemy]"

- Docs: prefect-sqlalchemy integration • API: API reference Example: configure pool on the connector

from prefect import flow, task
from prefect_sqlalchemy.database import SqlAlchemyConnector
from prefect_sqlalchemy.credentials import ConnectionComponents, SyncDriver
from sqlalchemy.pool import QueuePool

@task
def run_query(sql: str):
    connector = SqlAlchemyConnector(
        connection_info=ConnectionComponents(
            driver=SyncDriver.POSTGRESQL_PSYCOPG2,
            username="user",
            password="pass",
            host="db",
            port=5432,
            database="mydb",
        )
    )

    # IMPORTANT: reuse an Engine; avoid creating this in thousands of tasks if you can.
    engine = connector.get_engine(
        poolclass=QueuePool,
        pool_size=20,          # baseline pooled connections
        max_overflow=10,       # extra bursts allowed
        pool_timeout=60,       # wait longer if needed
        pool_pre_ping=True,    # drop stale conns proactively
        pool_recycle=1800      # recycle before server closes idle conns
    )

    # Always return connections to the pool
    with engine.begin() as conn:
        return conn.execute(sql).fetchall()

@flow
def main():
    # Submit tasks concurrently; see Prefect concurrency tips below
    futs = [run_query.submit("SELECT 1") for _ in range(25)]
    return [f.result() for f in futs]

Throttle DB usage with Prefect concurrency limits - Global limits (code-level): Global concurrency limits

from prefect.concurrency.sync import concurrency

@task
def run_query(sql):
    with concurrency("database", occupy=1, strict=True):
        ...

- Tag-based limits (orchestration-level): Tag-based concurrency limits

@task(tags=["database"])
def run_query(sql): ...

# Then set a limit equal to your pool capacity (or less):
prefect concurrency-limit create database 20

Checklist to avoid pool timeouts - Size pool to your true parallel demand and DB’s max connections (leave headroom for other services) - Use

pool_pre_ping=True

and an appropriate

pool_recycle

to handle stale/idle connections - Return connections with context managers (

engine.begin()

/

Session

scopes); don’t hold them across long operations - Reuse a single Engine per process instead of creating one per task - If using async drivers, use the async engine/session analogs and still apply concurrency limits in Prefect - Monitor server-side active connections (e.g., Postgres

pg_stat_activity

) to validate behavior If you’re not using Prefect blocks - Configure the SQLAlchemy engine similarly:

from sqlalchemy import create_engine
from sqlalchemy.pool import QueuePool

engine = create_engine(
    "<postgresql+psycopg2://user:pass@db:5432/mydb>",
    poolclass=QueuePool,
    pool_size=20,
    max_overflow=10,
    pool_pre_ping=True,
    pool_recycle=1800,
    pool_timeout=60,
)

A few clarifying questions so I can tailor this: - Which database and driver are you using (e.g., Postgres + psycopg2, MySQL + PyMySQL, etc.)?

- How are you creating/using the SQLAlchemy Engine (prefect-sqlalchemy

SqlAlchemyConnector

, direct

create_engine

, async)? - Roughly how many tasks/queries run concurrently at peak? - Are you creating an Engine per task, or reusing one per worker process? With those details, I can suggest concrete pool and concurrency settings that fit your setup.

Got it — since this is happening inside the Prefect API/Server, you’re hitting the Server’s own SQLAlchemy pool defaults: pool_size=5, max_overflow=10, pool_timeout=30. You can raise these via Prefect Server settings. What to configure (server-side) - pool_size: default 5 - max_overflow: default 10 - pool_timeout: default 30s - pool_recycle: default 3600s - pool_pre_ping: always enabled (hardcoded True), no config needed How to set them - Environment variables (most common in containers)

PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_SIZE=20
PREFECT_SERVER_DATABASE_SQLALCHEMY_MAX_OVERFLOW=40
PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_TIMEOUT=60
PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_RECYCLE=1800

- prefect.toml / pyproject.toml

[server.database.sqlalchemy]
pool_size = 20
max_overflow = 40
pool_timeout = 60.0
pool_recycle = 1800

Examples - Docker Compose

services:
  prefect-server:
    image: prefecthq/prefect:3-latest
    environment:
      PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_SIZE: "20"
      PREFECT_SERVER_DATABASE_SQLALCHEMY_MAX_OVERFLOW: "40"
      PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_TIMEOUT: "60"
      PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_RECYCLE: "1800"

- Helm (values override)

server:
  env:
    - name: PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_SIZE
      value: "20"
    - name: PREFECT_SERVER_DATABASE_SQLALCHEMY_MAX_OVERFLOW
      value: "40"
    - name: PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_TIMEOUT
      value: "60"
    - name: PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_RECYCLE
      value: "1800"

Operational tips - Each API/Server replica has its own pool. If you scale replicas, total DB connections = replicas × (pool_size + max_overflow). Make sure your Postgres max_connections can handle it. - If you see spikes, increasing pool_timeout can help the API wait for a free connection instead of timing out. - Consider PgBouncer in front of Postgres for high-traffic deployments. - Monitor Postgres with

pg_stat_activity

and adjust as needed. Docs and references - Settings reference: https://docs.prefect.io/v3/api-ref/settings-ref/ - Docker Compose self-hosting: https://docs.prefect.io/v3/how-to-guides/self-hosted/docker-compose/ - Helm chart deployment: https://docs.prefect.io/v3/advanced/server-helm/ - Helm chart repo (values): https://github.com/PrefectHQ/prefect-helm/tree/main/charts/prefect-server If you can share how you’re running Server (Docker Compose vs. Helm, number of replicas, DB type/version), I can suggest concrete values for your setup.