https://prefect.io logo
Title
r

Robert Hales

06/07/2021, 9:02 AM
Hi there having a slight issue with the ECS agent, StartFlowRun, and running in parallel. As the ECS agent registers and deregisters a task definition every time it starts a flow, when starting multiple flows in parallel I get the following AWS error:
An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Too many concurrent attempts to create a new revision of the specified family."
k

Kevin Kho

06/07/2021, 4:41 PM
Hey @Robert Hales, Prefect needs to register/deregister when it starts a Flow to be able to support changes in RunConfig. I am not sure it’s feasible to not register/deregister when the RunConfig is the same. I suggest you open a ticket for this.
r

Robert Hales

06/07/2021, 4:46 PM
Thanks for the reply @Kevin Kho, I will do đŸ™‚
Ah I seem to have found an already open ticket https://github.com/PrefectHQ/prefect/issues/4402
k

Kevin Kho

06/07/2021, 5:00 PM
Oh yeah that’s true. Feel free to chime in there