Oz Shaked
10/18/2024, 10:03 AMserve
function with the following code:
def serve_multiple_flows(list_of_flows: List[Flow], concurrent_limit: int = 10):
list_of_deployments = [flow.to_deployment(name=flow.name) for flow in list_of_flows]
serve(*list_of_deployments, limit=concurrent_limit)
However, I’ve noticed that all the sub-flows and tasks run on the same worker that got the Parent Flow, and the other workers aren't being utilized.
Is there a built-in solution to distribute the workload across workers? I'd like to avoid creating another subprocess to serve tasks using prefect.task_worker.serve
and calling them with delay()
from the sub-flows.
Thank you so much in advance!