William Ouellette
08/25/2025, 1:08 PMFlow run could not be submitted to infrastructure: The service has encountered an internal error. Please try again later
Marvin
08/25/2025, 1:08 PMMarvin
08/25/2025, 1:13 PMcurl http://<your-server-host>:<port>/api/health
Docs: Server health check
2) Inspect server logs around the failure time
- If running locally, start with more verbosity:
prefect server start --log-level DEBUG
- If running via Docker/Helm, check the API container/pod logs for stack traces, “database is locked”, “no such column”, or IntegrityError.
3) Ensure the database is migrated
- Run:
prefect server database upgrade -y
- If you’re on SQLite and see “database is locked”, reduce concurrent load or move to Postgres for reliability.
4) Verify version parity (server vs. workers and deployments)
- On each machine (server and worker hosts), run:
prefect version
- Keep versions aligned (ideally same minor/patch). Release notes: Prefect 3.x release notes
5) Confirm server services are enabled and running
- List services:
prefect server services list-services
- If needed, start them in-process (for debugging/local):
prefect server services start-services
6) Check work pools and workers
- Verify work pools exist and types are correct:
prefect work-pool ls -v
- Ensure workers are running and polling the right pool:
prefect worker start -p <your-pool-name>
- If you’re using Docker/Kubernetes workers, ensure the worker environment has the required extras installed (e.g., pip install "prefect[docker]"
or prefect[kubernetes]
).
Useful docs
- Server CLI: Self-hosted server CLI
- Health check API: Health check
If you can share a bit more, I can be more specific:
- How are you running the server (local prefect server start
, Docker, or Helm)? Which database (SQLite vs Postgres)?
- Prefect versions on server and workers (prefect version
output)
- Work pool type(s) and how your workers are started
- A few lines of server API logs (around the time a run fails) and any worker logs for a failing run
Those details usually pinpoint whether it’s a DB migration/lock, a services issue, or a version/plugin mismatch.William Ouellette
08/25/2025, 1:14 PM