https://prefect.io
Join Slack
<@ULVA73B9P> hi we added variables for prefect-server - name: PREFECT_SERVER_DATABASE_SQLALCHEMY_...
# marvin-ai
d
@Marvin hi we added variables for prefect-server - name: PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_SIZE value: 100 - name: PREFECT_SERVER_DATABASE_SQLALCHEMY_MAX_OVERFLOW value: 20 - name: PREFECT_SERVER_DATABASE_SQLALCHEMY_POOL_TIMEOUT value: 10 for avoid loop and timeout error , but cuurenly get timeout exec_wal error
m
🔄 Researching your answer... this may take a while 🔧 Using tool: 
research_prefect_topic
📊 Tools used: • 
check_cli_command
(1x) • 
display_callable_signature
(2x) • 
explore_module_offerings
(3x) • 
read_github_issues
(1x) • 
research_prefect_topic
(2x) • 
search_github_discussions
(1x)
b
@Denys Y can you provide more of the error message you're seeing? Are you using Postgres or SQLite for your database?
d
I use Postrgres aws rds, prefect server 3.4.8, timeout error appear under load when many tasks run
👍 1
Copy code
tr, pr = await connector
	2025-08-26 17:15:01.280	
  File "/usr/local/lib/python3.12/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
	2025-08-26 17:15:01.280	
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	2025-08-26 17:15:01.280	
    return await __connect_addr(params, True, *args)
	2025-08-26 17:15:01.280	
  File "/usr/local/lib/python3.12/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
	2025-08-26 17:15:01.280	
           ^^^^^^^^^^^^^^^^^^^^
	2025-08-26 17:15:01.280	
    conn = await _connect_addr(
	2025-08-26 17:15:01.280	
  File "/usr/local/lib/python3.12/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
	2025-08-26 17:15:01.280	
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	2025-08-26 17:15:01.280	
    return await connect_utils._connect(
	2025-08-26 17:15:01.280	
  File "/usr/local/lib/python3.12/site-packages/asyncpg/connection.py", line 2421, in connect
	2025-08-26 17:15:01.280	
Traceback (most recent call last):
	2025-08-26 17:15:01.280	
15:15:01.245 | ERROR   | prefect.server.services.failexpiredpauses - Unexpected error in: TimeoutError()
	2025-08-26 17:15:01.280	
TimeoutError
	2025-08-26 17:15:01.280	
    raise TimeoutError from exc_val
	2025-08-26 17:15:01.280	
  File "/usr/local/lib/python3.12/asyncio/timeouts.py", line 115, in __aexit__




	2025-08-26 17:15:01.280	
               ^^^^^^^^^^^^^^^^^^^^^^^
	2025-08-26 17:15:01.280	
    async with compat.timeout(timeout):
	2025-08-26 17:15:01.280	
  File "/usr/local/lib/python3.12/site-packages/asyncpg/connection.py", line 2420, in connect
	2025-08-26 17:15:01.280	
            ^^^^^^^^^^^^
b
What instance size is your RDS server?
d
actually it's aurora, we suppose it should be supported it's db.t4g.medium
I also saw heavy write load during load testing
b
Yes, write load is expected as we'll be inserting metadata about flows/tasks into the database.
The AWS 
t
series is a burstable instance type. This means you get a certain amount of CPU credits on an hourly basis. Once you exceed your available credits, your instance will be throttled. This is most likely why you're seeing slowness after a period of sustained DB writes. I would recommend you look at using something like 
db.c6gd.large
for your RDS instance type. This will give you the same CPU and memory resources as the 
db.t4g.medium
instance type, but the 
c
series instance does not burst and you receive a higher performance baseline.
d
ok, got it, thanks it's dev instance on prod we have more powerful instance
3 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by