Version Prefect 3.0.11 Two Kubernetes work pools; ...
# ask-community
f
Version Prefect 3.0.11 Two Kubernetes work pools; Parent- and sub-flow are in same work pool. In the parent flow: • mapped task (A) calls
run_deployment
with all default args (i.e.
as_subflow=True, timeout=None, idempotency_key=None, work_queue_name=None
) • (A) specifies timeout and retries. Expected behavior is that the task fails if
run_deployment
takes longer than task timeout. • Main flow then waits for all mapped tasks to finish:
Copy code
deployment_future = run_deployment_task.map(arg1 = [1, 2, 3], arg2 = unmapped(my_constant))
deployment_results = deployment_future.result()
for result in deployment_results:
    if result.state_type != StateType.COMPLETED:
      	all_sub_flows_succeeded = False
       	break
...
With task A we are triggering a number of flow runs and what we observe is, that all the sub-flows finish with status COMPLETED. However, one out of some 30 of the instances of A will not return as the
run_deployment
call never finishes. Now there are two questions: 1. How can we investigate why
run_deployment
does not return? Does it loose track of the flow run it creates? 2. Why is the timeout that task A defines not kicking in? Any thoughts and Ideas are HIGHLY appreciated! And of course please feel free to point out all the stuff I failed to mention.... All the best, Florian
BTW, here is how it looks on the prefect UI:
All - we've found that this issue seems to subside when we set
as_subflow=False
. Unfortunately I don't understand why this would be but should the whole use case be an indicator for a but in prefect, maybe it is a hint?