Fredrik Hoem Grelland
11/13/2025, 1:30 PMMarvin
11/13/2025, 1:30 PMMarvin
11/13/2025, 1:43 PMtask.submit(...), everything is happening in the same Python process using Prefect’s default thread pool. You should attach your debugger to the flow process itself (not a separate worker), because submitted tasks run in threads created by the in‑process task runner.
What to do to see why a submitted task is still pending:
- Attach your debugger to the flow process before you call task.submit(...). Since tasks run in threads, once attached you can hit breakpoints in your flow, in the task function, and in Prefect internals.
- If you’re using VS Code, you can embed:
import debugpy
debugpy.listen(("0.0.0.0", 5678))
debugpy.wait_for_client() # start your debugger and attach here
- Inspect the future’s state right after submit:
f = my_task.submit(...)
st = f.state
print(st.type, st.name, st.message)
print(st.state_details) # scheduled_time, wait_for, etc.
This will usually tell you exactly why it’s not running yet (e.g., scheduled in the future, waiting on dependencies, waiting for a concurrency slot).
- If you want deeper detail, read the full task run with the client:
from prefect.client.orchestration import get_client
with get_client(sync_client=True) as client:
tr = client.read_task_run(task_run_id=f.task_run_id)
print(tr.state.type, tr.state.message, tr.state.state_details)
Common reasons a submitted task remains pending:
- It’s SCHEDULED for a future scheduled_time.
- It’s waiting on upstreams via wait_for=....
- It’s waiting for a tag-based concurrency slot.
- All thread-pool workers are busy (e.g., max_workers too low) and your flow is blocking on other .result() calls.
Where to put breakpoints:
- In your flow: immediately after task.submit(...) to inspect f.state.
- In the task function itself (top of the function) — if the task never starts, you won’t hit this, which confirms it’s stuck before execution.
- In Prefect internals that govern state transitions and submission:
- Accessing f.state goes through Prefect’s state retrieval; stepping into that property lets you see the current orchestration reason (pending/scheduled/concurrency).
- The in-process submission path is handled by the thread-based task runner (default is ThreadPoolTaskRunner). Once attached to the process, breakpoints in the task runner submission path will hit because it’s just threads.
Helpful knobs and references:
- Turn on debug logs to see scheduling/concurrency messages:
PREFECT_LOGGING_LEVEL=DEBUG
- Default runner and concurrency overview: Task runners and Run work concurrently
- State details and what “Pending” means: States
- Concurrency limits (tag-based): Tag-based concurrency limits
- Logging customization: Logging customization
If you can share:
- The output of f.state (including state_details)
- Whether you’re using tags with limits
- Your task runner config (e.g., ThreadPoolTaskRunner(max_workers=...))
I can help pinpoint exactly where to set a breakpoint or what to tweak next.