@Marvin when I start my prefect workers with

--with-healthcheck

I get this exception when running a job

Failed to submit flow run '3c34553f-c466-447d-b032-f2b22e976c7d' to infrastructure.
Traceback (most recent call last):
  File "/app/.venv/lib/python3.13/site-packages/prefect/workers/base.py", line 1288, in _submit_run_and_capture_errors
    result = await self.run(
             ^^^^^^^^^^^^^^^
    ...<3 lines>...
    )
    ^
  File "/app/.venv/lib/python3.13/site-packages/prefect/workers/process.py", line 256, in run
    raise RuntimeError("Failed to start flow run process.")
RuntimeError: Failed to start flow run process.

✅ thought for 366.0 seconds

🤨

thanks! taking a look at this (very weird traceback imo)

@Nate I will DM you the entire trace

oh interesting, so i think this is a 3.13 thing

Thanks for the stack trace — that’s coming from the Process worker trying (and failing) to spawn the subprocess that actually runs your flow, then raising the generic “Failed to start flow run process.” error. A couple of important notes and questions to help pinpoint this: - You’re on Python 3.13 (based on the path). Prefect’s workers and their subprocess handling are heavily tied to asyncio/anyio internals; Python 3.13 introduced changes that have bitten similar code paths in the past. Can you confirm your Prefect version with:

prefect version

- Does the exact same deployment run successfully if you start the worker without

--with-healthcheck

? - What platform/base image are you on (e.g., Debian/Ubuntu vs. Alpine; Docker tag), and are you running inside a container? - Could you share the worker logs around the failure with debug logs enabled?

PREFECT_LOGGING_LEVEL=DEBUG prefect worker start --pool <POOL_NAME> --with-healthcheck

Why

--with-healthcheck

matters - That flag starts a tiny uvicorn server in the worker process to expose a /health endpoint (prefect/workers/server.py). In some environments, running an async web server alongside spawning subprocesses can clash with event-loop/signal handling and cause the underlying process creation to fail — which then surfaces as the generic RuntimeError you’re seeing. - There’s a related (though not identical) symptom tracked when a web server is enabled for runners that ends up breaking subprocess handling due to event loop/signal watcher issues: #15733. Your case is with workers + healthcheck, but the failure mode (“Failed to start process…”) is similar. Recommended next steps (to unblock) 1) Quick isolation test - Start the worker without the healthcheck:

prefect worker start --pool <POOL_NAME>

If flows run fine, the healthcheck server is the trigger. 2) Try a supported runtime - If you’re on Python 3.13, try downgrading the worker runtime to Python 3.12 or 3.11 (e.g., the official images are tested:

prefecthq/prefect:3-python3.12

). This is the fastest way to rule out a Python 3.13 runtime issue. - Prefect source for the Process worker: prefect/workers/process.py 3) Gather more detail - Re-run with debug logs (above) and share the lines just before the RuntimeError — there’s usually a lower-level exception that tells us exactly why the subprocess failed (e.g., an asyncio/signal handler or child-watcher issue). If you need a health endpoint now - Temporarily run the worker without

--with-healthcheck

and use a simple sidecar health endpoint (e.g., a tiny HTTP server that returns 200) for Kubernetes liveness/readiness until we confirm Python/runtime compatibility. It won’t reflect worker queue state, but it’ll keep your pods managed while we sort the root cause. - Alternatively, pin to a known-good base (e.g.,

prefecthq/prefect:3-python3.12

) and keep

--with-healthcheck

enabled there. If you can share: - The output of

prefect version

- Whether removing

--with-healthcheck

fixes it - Your OS/base image - A short DEBUG log excerpt surrounding the failure …I can zero in on a precise fix or file a targeted issue/PR if it’s a 3.13-specific incompatibility.

issue created for tracking! thanks for raising @Jordan Jones

I don't have it explicitly installed in my pyproject.toml but I do see it in my uvlock. When I exec into the container and open REPL and try to import it I get

no module named 'uvloop'

Also I downgraded to python 3.12.12 and get the same crashing

Also I downgraded to python 3.12.12 and get the same crashing

when you say you downgraded, how did you downgrade? ie you changed the process running

prefect worker start

and not for the flow run process right?

Yeah, I updated my dockerfile

FROM

image that the prefect worker uses. If I exec into my prefect worker container and

python --version

I get

3.12.12

hrm ok looking

Also confused on why it doesn't fail on startup of the worker. Maybe I am miss-understanding how it works?

can you send your dockerfile by chance?