< Marvin> I ve got a windows process worker v3 6 23 that get Prefect Community #marvin-ai

<@ULVA73B9P> I've got a windows process worker (v3...

Jordan Fisher

05/12/2026, 7:33 AM

@Marvin I've got a windows process worker (v3.6.23) that gets stuck and unable to submit new flows when a timeout error happens. It appears the engine is having some sort of internal error when trying to load telemetry? Here's the stack trace, is this is a known bug / can this be resolved?

Copy code

07:36:48.665 | ERROR   | prefect.engine - Execution of flow run '06a01fab-5393-7740-8000-4194d1dc0c49' exited with unexpected exception
Traceback (most recent call last):
  File "C:\Python314\Lib\site-packages\prefect\telemetry\_metrics.py", line 79, in RunMetrics
    from opentelemetry.exporter.otlp.proto.http.metric_exporter import (
        OTLPMetricExporter,
    )
ModuleNotFoundError: No module named 'opentelemetry.exporter'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python314\Lib\site-packages\prefect\engine.py", line 57, in handle_engine_signals
    yield
  File "C:\Python314\Lib\site-packages\prefect\engine.py", line 124, in <module>
    run_coro_as_sync(
    ~~~~~~~~~~~~~~~~^
        run_flow(flow, flow_run=flow_run, error_logger=run_logger)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )

Marvin

05/12/2026, 7:33 AM

✅ thought for 242.2 seconds

Marvin

05/12/2026, 7:37 AM

This does look like a real issue, and your traceback suggests telemetry is getting invoked while Prefect is already handling a flow failure/timeout, then hitting a missing dependency:

Copy code

No module named 'opentelemetry.exporter'

I did not find an existing public GitHub issue/discussion for this exact Windows + process worker + timeout + stuck-worker combination, so I can’t say “yes, this is a known tracked bug” with confidence. But based on the code path, it’s very plausible that this is a Prefect bug or edge case in

3.6.23

, especially on Windows. What stands out: - The failure is happening inside

prefect.telemetry._metrics.RunMetrics

- It’s triggered while the engine is unwinding an exception - After that, your process worker becomes unable to submit new runs That strongly suggests the worker/engine cleanup path is being disrupted by telemetry initialization or shutdown. Most likely root cause Prefect 3.6.23 enables resource metrics collection by default for flow run subprocesses. That code attempts to import OpenTelemetry exporters. If those packages are unavailable in the runtime where the flow run executes, you can get exactly this kind of import failure. Your traceback:

Copy code

No module named 'opentelemetry.exporter'

means the runtime where the flow run is executing does not have the OTel exporter package available, but Prefect is still reaching a telemetry code path during failure handling. What I’d recommend first 1. Disable resource metrics telemetry on that worker 2. Upgrade off 3.6.23 if you can I’d start with telemetry disablement because it’s the fastest way to remove the failing code path. Set this in the worker environment before starting the worker:

Copy code

PREFECT_TELEMETRY_ENABLE_RESOURCE_METRICS=false

On Windows CMD:

Copy code

set PREFECT_TELEMETRY_ENABLE_RESOURCE_METRICS=false
prefect worker start --pool YOUR_POOL

On PowerShell:

Copy code

$env:PREFECT_TELEMETRY_ENABLE_RESOURCE_METRICS="false"
prefect worker start --pool YOUR_POOL

I verified the worker CLI syntax exists:

Copy code

prefect worker start --help

If you manage the worker as a Windows service / scheduled task / wrapper script, put that env var there so it’s always present. Why this should help That setting disables OS-level resource metric collection in flow run subprocesses, which should bypass the

RunMetrics

path that’s trying to import:

Copy code

opentelemetry.exporter.otlp.proto.http.metric_exporter

Should you install OpenTelemetry instead? Possibly, but I’d treat that as secondary. Since the bigger symptom is “worker gets stuck after timeout”, simply satisfying the import may not fully solve the underlying robustness issue. Still, if you want to test it, check whether your flow execution environment includes the OTel exporter package. The missing module is typically provided by:

Copy code

opentelemetry-exporter-otlp-proto-http

But again: my stronger recommendation is disable telemetry first, then verify whether the worker recovers cleanly after timeouts. Version recommendation I’d also recommend upgrading from

3.6.23

to at least the latest

3.6.x

, and ideally

3.7.x

if that fits your environment. I found current release notes here: Prefect 3.7.0 release notes I didn’t find a changelog entry for this exact bug, but there have been multiple worker/timeout/lifecycle improvements after

3.6.23

, so you’re on a version old enough that upgrading is worthwhile. Practical next steps I’d test in this order: 1. Restart the worker with:

Copy code

PREFECT_TELEMETRY_ENABLE_RESOURCE_METRICS=false

2. Reproduce the timeout 3. Confirm whether the worker continues picking up new flow runs afterward 4. If yes, upgrade Prefect next 5. If it still gets stuck, then this is likely a broader Windows process-worker timeout bug, not just telemetry If you want to keep telemetry enabled

Marvin

05/12/2026, 7:37 AM

Then ensure the worker’s Python environment has the needed package installed in the same environment the flow run subprocess uses. Something like:

Copy code

pip install opentelemetry-exporter-otlp-proto-http

But again, I’d only do this after testing telemetry-off first. My bottom line - Yes, this looks like a legitimate Prefect bug/edge case - No, I couldn’t find a publicly tracked known issue for this exact symptom - Most likely workaround: disable resource metrics telemetry - Best longer-term fix: upgrade off

3.6.23

If you want, I can also help you with a minimal reproduction plan for Windows process workers so you can confirm whether this is specifically timeout-related or tied to your worker environment.

Jordan Fisher

05/12/2026, 7:56 AM

@Marvin After disabling telemetry the module is still attempting to load. I've also upgraded to 3.7.0 which hasn't changed anything.

Marvin

05/12/2026, 7:56 AM

🔄 Researching your answer... this may take a while 🔧 Using tool:

explore_module_offerings

📊 Tools used: •

explore_module_offerings

(2x) •

research_prefect_topic

(1x)

Marvin

05/12/2026, 8:00 AM

Thanks — that’s actually a very important clue. If disabling

PREFECT_TELEMETRY_ENABLE_RESOURCE_METRICS

still results in Prefect trying to import the telemetry metrics module, then one of two things is happening: 1. the setting is not reaching the flow-run subprocess, or 2. this traceback is coming from a code path before the setting gate is applied. Based on the code, the env var should short-circuit before the OpenTelemetry import. So if you still see:

Copy code

No module named 'opentelemetry.exporter'

then I would not assume “telemetry disablement is broken globally”; I’d assume the subprocess executing the flow is not seeing the setting you set on the worker. Important distinction There are two different things here: - importing

RunMetrics

itself -

RunMetrics

trying to import

opentelemetry.exporter

The first can still happen harmlessly. The second should not happen if

PREFECT_TELEMETRY_ENABLE_RESOURCE_METRICS=false

is actually loaded in that subprocess. So the fact that you still get the

opentelemetry.exporter

error strongly suggests a settings propagation/environment mismatch. Most likely cause in your case You mentioned: - Windows - process worker - timeout path - Python 3.14 - worker gets stuck The suspicious bit is Python 3.14. That is not a common/typical runtime target for Prefect yet, and I would consider it a major variable here. If the worker process and the spawned flow-run subprocess are not using the exact same environment/interpreter layout, you can absolutely get behavior like: - worker sees your env var - subprocess does not - subprocess also lacks the optional OTel package - exception path then wedges the worker So at this point I’d focus less on telemetry config itself and more on what environment the flow subprocess is actually starting with. What to test next 1. Verify the setting inside the actual flow-run process Add a tiny debug flow that logs the env var and Prefect setting value:

Copy code

python
import os
from prefect import flow, get_run_logger
from prefect.settings import get_current_settings

@flow
def debug_settings():
    logger = get_run_logger()
    <http://logger.info|logger.info>(
        f"PREFECT_TELEMETRY_ENABLE_RESOURCE_METRICS env = "
        f"{os.getenv('PREFECT_TELEMETRY_ENABLE_RESOURCE_METRICS')}"
    )
    <http://logger.info|logger.info>(
        f"settings.telemetry.enable_resource_metrics = "
        f"{get_current_settings().telemetry.enable_resource_metrics}"
    )

if __name__ == "__main__":
    debug_settings()

If that logs

True

None

, then the subprocess is not inheriting your setting. 2. Check how the worker is launched on Windows If it’s started from: - Task Scheduler - NSSM / Windows service - a batch file - PowerShell profile - another supervisor the env var may not actually exist in the worker’s process environment the way you expect. 3. Set the env var at the machine/user level temporarily Instead of only in the current shell, try setting it persistently and then start a fresh worker. PowerShell:

Copy code

[System.Environment]::SetEnvironmentVariable(
  "PREFECT_TELEMETRY_ENABLE_RESOURCE_METRICS",
  "false",
  "Machine"
)

Then fully restart the worker host/session. 4. Also check for conflicting Prefect profiles/settings A profile or config file could be overriding what you expect. The flow snippet above is the fastest way to tell. Strong recommendation: try Python 3.12 or 3.11 Honestly, this is the biggest red flag in your traceback:

Copy code

C:\Python314\Lib\site-packages\prefect\...

I would strongly recommend testing the same worker on Python 3.11 or 3.12. Even if Prefect installs on 3.14, this kind of subprocess/timeout/import edge case is exactly the kind of thing that can show up first on a very new Python runtime. So my prioritized advice now is: 1. Move the worker to Python 3.11/3.12 2. Keep Prefect on 3.7.0 or latest 3.6.x/3.7.x 3. Verify the telemetry setting from inside the flow subprocess

Marvin

05/12/2026, 8:00 AM

4. If needed, install the missing OTel exporter package as a temporary guardrail Temporary guardrail Even though it shouldn’t be necessary if telemetry is truly disabled, you can test whether this removes the crash path by installing:

Copy code

pip install opentelemetry-exporter-otlp-proto-http

If that stops the traceback but the worker still wedges on timeout, then telemetry was only the symptom, not the real root cause. What I think is most likely now Given your update, my current best guess is: - this is not just “telemetry didn’t turn off” - it’s likely a subprocess environment inconsistency and/or Python 3.14 compatibility issue - the telemetry import failure is what surfaces first during exception handling Most useful next data point If you can share the output of that

debug_settings()

flow from a run executed by the worker, I can help narrow it down very quickly. If you want, I can also give you a minimal timeout reproduction flow plus a small checklist to determine whether the worker process itself is wedged or only the flow subprocess is failing.

Jordan Fisher

05/12/2026, 8:22 AM

@Marvin Ah it turns out its not related to the telemetry, the engine ins't getting stuck, it appears that even though the task times out, the subprocess that it spawned is continuing to run beyond the timeout and is causing everything to hang until it finished. Whats the best way to ensure the subprocess is killed when the task times out?

Marvin

05/12/2026, 8:22 AM

✅ thought for 134.3 seconds

Marvin

05/12/2026, 8:25 AM

Yes — that matches Prefect’s timeout semantics. A Prefect task timing out does not automatically kill OS subprocesses spawned by your task. On Windows especially, the timeout cancels the Python task context, but any child process you launched with

subprocess

can keep running unless you terminate it. So the best fix is: don’t rely on Prefect task timeout to kill subprocesses; give the subprocess its own timeout and cleanup logic. Best options 1. For simple sync code: use
subprocess.run(..., timeout=...)
This is the easiest and usually best approach if you just need to run one command and fail if it runs too long.

Copy code

python
import subprocess
from prefect import task

@task
def run_command():
    try:
        result = subprocess.run(
            ["your-command", "arg1", "arg2"],
            capture_output=True,
            text=True,
            timeout=60,
            check=True,
        )
        return result.stdout
    except subprocess.TimeoutExpired:
        raise RuntimeError("Subprocess exceeded 60s timeout and was terminated")

That timeout is enforced by Python’s subprocess machinery, not Prefect’s task cancellation, so it’s much more reliable for this case. 2. If you need
Popen
, kill it explicitly in
finally
Use this when you need streaming output, polling, or more control.

Copy code

python
import os
import signal
import subprocess
import sys
import time
from prefect import task

@task
def run_command():
    proc = subprocess.Popen(
        ["your-command", "arg1"],
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        text=True,
        creationflags=subprocess.CREATE_NEW_PROCESS_GROUP if sys.platform == "win32" else 0,
    )

    try:
        stdout, stderr = proc.communicate(timeout=60)
        return stdout
    except subprocess.TimeoutExpired:
        if sys.platform == "win32":
            proc.kill()
        else:
            proc.kill()
        raise RuntimeError("Subprocess exceeded 60s timeout and was killed")
    finally:
        if proc.poll() is None:
            proc.kill()

For many cases on Windows,

proc.kill()

is the most dependable option. 3. If the subprocess itself spawns children, use Windows
taskkill
This is often the real answer on Windows.

proc.kill()

may kill only the parent, while grandchildren continue running. If your command launches its own subprocess tree, use:

Copy code

python
import subprocess
from prefect import task

@task
def run_command():
    proc = subprocess.Popen(
        ["your-command"],
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        text=True,
    )

    try:
        stdout, stderr = proc.communicate(timeout=60)
        return stdout
    except subprocess.TimeoutExpired:
        subprocess.run(
            ["taskkill", "/PID", str(proc.pid), "/T", "/F"],
            capture_output=True,
            text=True,
        )
        raise RuntimeError("Subprocess tree exceeded timeout and was force-killed")

On Windows,

/T /F

is usually the most robust way to kill the whole process tree. Recommendation: if your timed-out task is spawning another executable that may itself spawn more processes, use
taskkill /T /F
. 4. Prefer subprocess-level timeout over Prefect task timeout Instead of:

Copy code

python
@task(timeout_seconds=60)
def my_task():
    subprocess.Popen(...)

prefer:

Copy code

python
@task
def my_task():
    subprocess.run(..., timeout=60)

You can still keep a Prefect task timeout as a backup, but the subprocess timeout should be the primary control. Good production pattern For Windows, I’d structure it like this: ```python import subprocess from prefect import task @task def run_external_job(): proc = subprocess.Popen( ["your-command", "arg1"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, ) try: stdout, stderr = proc.communicate(timeout=300) if proc.returncode != 0: raise RuntimeError(

Marvin

05/12/2026, 8:25 AM

f"Command failed with code {proc.returncode}\n" f"stdout\n{stdout}\n\nstderr\n{stderr}" ) return stdout except subprocess.TimeoutExpired: subprocess.run( ["taskkill", "/PID", str(proc.pid), "/T", "/F"], capture_output=True, text=True, ) raise RuntimeError("Command timed out and the process tree was killed")

Copy code

*Important caveat*

If your task uses the default `ThreadPoolTaskRunner` and is doing blocking sync work, Prefect’s timeout won’t interrupt that blocking call cleanly anyway. That’s another reason to manage the subprocess timeout directly.

So the practical rule is:

- *Use Prefect timeout for task state/orchestration*
- *Use subprocess timeout/process-kill logic for external commands*

*Bottom line*

On Windows, the best way to ensure cleanup is usually:

- launch subprocess normally
- enforce timeout with `communicate(timeout=...)` or `subprocess.run(timeout=...)`
- on timeout, call:
  ```taskkill /PID <pid> /T /F

If you want, I can help you turn your current task code into a safe Windows wrapper that: - captures logs, - times out cleanly, - kills the full process tree, - and raises a Prefect-friendly failure.

Open in Slack

Previous Next