progre55
07/02/2025, 1:53 PMTimeoutError
. What are my options? How can I increase the database timeout, or should I delete the flow run manually somehow?Marvin
07/02/2025, 1:53 PMMarvin
07/02/2025, 1:54 PMPREFECT_API_DATABASE_TIMEOUT
environment variable to a higher value (in seconds) before starting your Prefect server:
bash
export PREFECT_API_DATABASE_TIMEOUT=300 # 5 minutes
2. Database-Level Solution
If you're using RDS PostgreSQL, you can also adjust the statement timeout at the database level:
sql
ALTER DATABASE your_prefect_db SET statement_timeout = '300000'; -- 300 seconds (5 minutes)
Or for the current session only:
sql
SET statement_timeout = '300000';
3. Manual Deletion
If the UI deletion continues to fail, you can delete the flow run directly from the database. However, this should be done with extreme caution as it might leave orphaned data. Here's the general approach:
sql
-- First, identify the flow run
SELECT * FROM flow_run WHERE id = 'your-flow-run-id';
-- Then delete it (be very careful with this!)
DELETE FROM flow_run WHERE id = 'your-flow-run-id';
Important Considerations:
- Before making any direct database changes, always backup your database
- If you're running in a containerized environment, make sure to set the environment variable in your container configuration
- Consider investigating why this particular deletion is timing out - it might indicate a larger issue with database performance or size
- If you're experiencing frequent timeouts, you might want to implement a regular cleanup strategy to prevent the database from growing too large
The safest approach is to start with increasing the PREFECT_API_DATABASE_TIMEOUT
environment variable. If that doesn't resolve the issue, you may want to investigate the size of your database and consider implementing a retention policy to automatically clean up old flow runs.
For more information about self-hosted Prefect server configuration, you can refer to the official documentation.progre55
07/03/2025, 9:28 AMprogre55
07/03/2025, 9:28 AMMarvin
07/03/2025, 9:28 AMMarvin
07/03/2025, 9:29 AMPREFECT_API_DATABASE_TIMEOUT
is not explicitly set. This is quite conservative and might be insufficient for larger databases or operations involving many records, which explains why you're experiencing timeout issues when trying to delete flow runs.
Given this relatively short default timeout, it's not uncommon to need to increase this value in production environments, especially when working with self-hosted instances that may have accumulated significant amounts of data.