Saad Saeed
10/09/2025, 10:22 PMClient error '429 Too Many Requests' for url '<https://api.prefect.cloud/api/accounts/014c1c49-a193-4f08-ac27-a269c0bdf2c5/workspaces/0f135d37-f0c4-4a7b-bd86-af19e8547f54/logs/>'
Response: {'detail': 'API rate limit reached. To view rate limit usage, visit <https://app.prefect.cloud/organization/014c1c49-a193-4f08-ac27-a269c0bdf2c5/rate-limits>. To raise your rate limit please contact <mailto:help@prefect.io|help@prefect.io>'}
Marvin
10/09/2025, 10:23 PMMarvin
10/09/2025, 10:24 PMINFO
or WARNING
) and quiet noisy libraries.
- Avoid logging large payloads; summarize instead.
- Tune log batching to lower request rate:
- Increase the batch interval so logs are sent less frequently.
- Cap max log size to avoid oversized log records.
- Stagger/limit concurrency to smooth bursts.
- Allow the client to retry/back off on 429s by increasing max retries.
Suggested settings (env vars)
- Disable logs to API temporarily (for heavy jobs) while you investigate:
- PREFECT_LOGGING_TO_API_ENABLED=false
- Otherwise, tune batching and size:
- PREFECT_LOGGING_TO_API_BATCH_INTERVAL=5
(seconds; increase from default ~2s)
- PREFECT_LOGGING_TO_API_MAX_LOG_SIZE=200000
(bytes; trims very large messages)
- Reduce verbosity:
- PREFECT_LOGGING_LEVEL=INFO
(or WARNING
)
- Increase HTTP client resilience:
- PREFECT_CLIENT_MAX_RETRIES=8
(default is 5)
Example
export PREFECT_LOGGING_LEVEL=INFO
export PREFECT_LOGGING_TO_API_ENABLED=true
export PREFECT_LOGGING_TO_API_BATCH_INTERVAL=5
export PREFECT_LOGGING_TO_API_MAX_LOG_SIZE=200000
export PREFECT_CLIENT_MAX_RETRIES=8
If you can share a bit more about your workload (approx concurrent runs, logging level, whether you recently enabled DEBUG or added large logs), I can suggest a tighter set of changes.