<@ULVA73B9P> [{'title': 'Prefect flow run notifica...
# ask-marvin
a
@Marvin [{'title': 'Prefect flow run notification - Failed', 'text': "Flow run Per Farm and Field Parallel Pipeline Flow/lurking-dragon observed in state
Failed
at 2026-03-19 111548.407635+00:00. Flow ID: a92567ba-5cfe-4577-b486-12b0e2f116b2 Flow run ID: 82442e05-843b-4461-85bc-46315c1e9830 Flow run URL: State message: Flow run encountered an exception: PrefectHTTPStatusError: Server error '500 Internal Server Error' for url 'http://prefect-server/api/concurrency_limits/decrement' Response: {'exception_message': 'Internal Server Error'} Why do i see this error
m
thought for 240.9 seconds
Thanks for sharing the notification. Here’s what that error means and how to track it down. What’s happening - That endpoint (
/api/concurrency_limits/decrement
) is called by the server when a flow (or tasks with concurrency tags) finishes so Prefect can release any occupied concurrency slots. - A 500 Internal Server Error means the Prefect Server hit an exception on its side (not your flow code) while trying to release those slots. Common causes - Database/migrations mismatch or DB connectivity issues (e.g., SQLite “database is locked”, Postgres timeouts) - A problematic or stale concurrency limit/lease record (e.g., limit was deleted or is inconsistent while a flow tries to release it) - An older Prefect Server version with a bug that’s been fixed in a newer release What to check next 1) Grab the server logs around 2026-03-19 111548 UTC - This will have the traceback that explains the 500. Please share the stack trace here so I can pinpoint it. 2) Confirm your Prefect version (and consider updating)
Copy code
prefect version
pip install --upgrade prefect
3) Ensure your server DB is up-to-date (run where the server is installed)
Copy code
prefect server database upgrade
4) Inspect your concurrency limits
Copy code
prefect concurrency-limit ls
prefect concurrency-limit inspect <tag>
- If one tag/limit looks off, try resetting or recreating it:
Copy code
prefect concurrency-limit reset <tag>
# or
prefect concurrency-limit delete <tag>
prefect concurrency-limit create <tag> --limit <N>
5) If you’re using SQLite in a multi-worker setup, consider Postgres for reliability. Quick isolation step - Temporarily remove concurrency tags from the affected flow/tasks or pause/delete the specific concurrency limit to see if the failures stop. If they do, we’ve confirmed it’s tied to that limit/lease state. Helpful docs - Concurrency limits overview: Concurrency limits in Prefect - CLI reference (concurrency limits): run
prefect concurrency-limit --help
If you can share: - Prefect version - Backing DB (SQLite/Postgres) and how the server is deployed - The server logs/traceback for that 500 around the timestamp …I can give you a targeted fix.