Fugo Takefusa
10/02/2025, 11:25 PMcurl https://<prefect-server-url>/api/admin/version)Marvin
10/02/2025, 11:25 PMMarvin
10/02/2025, 11:33 PM.fn (e.g., my_task.fn(...)) or call the underlying function via a wrapper, Prefect does not create a task run — it just executes Python code. This is the most common cause of “flows run but no task runs.”
- Also check for additional decorators around your tasks/flows. In Prefect 3.x, the Prefect decorator generally needs to be the outermost decorator. Wrapping a @task or @flow with your own decorator can strip the Prefect instrumentation and lead to an empty graph and missing logs (very similar to https://github.com/PrefectHQ/prefect/issues/16182).
- Good:
@task(log_prints=True)
def do_thing(): ...
@flow
def my_flow():
do_thing() # or do_thing.submit()
- Risky:
@my_decorator
@task
def do_thing(): ...
@my_decorator
@flow
def my_flow(): ...
- If you must decorate, ensure @task / @flow is outermost and your decorator uses functools.wraps properly.
2) Verify Prefect versions in ALL containers match (server, worker, and the image that runs your flows)
- A version mismatch can break event publishing even if runs otherwise “work”.
- Please run prefect version inside:
- the API server container
- the worker container
- the flow execution container/image in ECS
- Make sure they’re all on the same 3.7.20 or, preferably, upgrade all of them to the same newer patch/minor. If you can, try the latest 3.8.x+ across the board; there have been numerous event/logging robustness fixes since early 3.x.
3) Check worker/execution logs for event publishing failures
- Look for errors like:
- “EventsWorker failed with X pending items”
- “Could not publish event” / httpx errors
- If you see these, the flow engine tried to create task runs, but event publishing failed. This can be caused by:
- transient networking through an ALB/ingress
- TLS/HTTP2 oddities
- client/server version mismatch
- To get more detail, set PREFECT_LOGGING_LEVEL=DEBUG in the worker/execution environment and re-run a small flow.
4) Minimal, known-good test on your stack
- Deploy/run this minimal flow unchanged on the same infrastructure:
from prefect import flow, task
@task(log_prints=True)
def t(name):
print(f"hello from {name}")
@flow(log_prints=True)
def demo():
t("call") # creates a task run
t.submit("sub") # also creates a task run
if __name__ == "__main__":
demo()
- You should see two task runs under the flow run. If you do not, it’s almost certainly an event publishing / instrumentation / version issue rather than your business logic.
5) Sanity-check API URL and server logs
- Since your flow runs are appearing, PREFECT_API_URL is likely correct. Still confirm the exact value inside the execution container matches the server you queried with curl.
- On the API server, check logs around the time of your runs for “event ingestion” or “graph” errors.
If none of the above resolve it, could you share the following so we can pinpoint this quickly?Marvin
10/02/2025, 11:33 PMprefect version from each)
- Any related worker/execution logs, especially mentions of “EventsWorker” or httpx errors
- Whether your tasks are ever called via .fn or wrapped by another decorator
Why I suspect instrumentation or versioning here
- Your description matches two known patterns: tasks running as plain Python (due to .fn or wrapping decorators) and event publishing failures (e.g., “EventsWorker failed…” in #15153).
- You’re on a later 3.7.x server, but the execution image may not be. When the client and server are out of sync, the event pathway that creates task runs can break even if the flow succeeds.
References
- Issue about missing task runs in 3.x: https://github.com/PrefectHQ/prefect/issues/15153
- Decorator-related empty graph/no task runs: https://github.com/PrefectHQ/prefect/issues/16182
- Settings and profiles: https://docs-3.prefect.io/v3/concepts/settings-and-profiles
- Server concepts: https://docs-3.prefect.io/v3/concepts/server
If you can share the snippets and versions, I’m happy to help you zero in on the exact cause.