Ben Muller
11/18/2022, 12:51 AMrun_deployment
to essentially launch 30 concurrent ecs tasks. What I notice is that 5 of the deployment runs start and the other 25 are pending.
How do I make this behave similar to prefect 1.0 where I was able to have all of these run at the same time ? Is this a limitation of my agent ? I was under the impression all the agent does is orchestrate the ecs tasks in my aws environment?Anna Geller
Ben Muller
11/18/2022, 12:56 AMfrom prefect.deployments import run_deployment
for start in range(16000, 76000, 2000):
end = start + 2000
run_deployment(
name="my-flow/default",
flow_run_name=f"my-flow{start}:{end}",
parameters=dict(start=start, end=end),
timeout=0,
)
prefect work-queue clear-concurrency-limit 'prod'
Anna Geller
Ben Muller
11/18/2022, 1:11 AMAnna Geller
Ben Muller
11/18/2022, 1:13 AMAnna Geller
Ben Muller
11/18/2022, 1:21 AMAnna Geller
Ben Muller
11/18/2022, 1:22 AMAnna Geller
Ben Muller
11/18/2022, 1:39 AMAnna Geller
Ben Muller
11/18/2022, 1:40 AMAnna Geller
Ben Muller
11/18/2022, 1:42 AMAnna Geller
Ben Muller
11/18/2022, 2:01 AMAnna Geller
Ben Muller
11/18/2022, 4:29 AM0 * * * *
there appears like maybe there is a race condition with the agent/queue and the flows that dont get in first, stay in the Pending
state indefinitely.
I just logged onto my UI and I have flows that are up to 3 hours late and just never get started.Anna Geller
Ben Muller
11/18/2022, 4:34 AMAnna Geller
Ben Muller
11/18/2022, 4:35 AMAnna Geller
Ben Muller
11/18/2022, 4:40 AM