@Marvin how can I avoid this crash type when submitting flow-runs to fargate "Flow run could not be submitted to infrastructure: An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Too many concurrent attempts to create a new revision of the specified family."
m
Marvin
03/28/2024, 4:01 PM
To avoid the crash when submitting flow runs to Fargate due to too many concurrent attempts to create a new revision of the specific family, consider the following strategies:
1. Allocate enough CPU and memory to your agent to make sure it can handle provisioning infrastructure for your flow runs.
2. Set a work-queue concurrency limit to prevent the agent from processing all runs simultaneously, which can lead to resource contention and API call errors.
Keep in mind that some API call failures could be due to transient client or server-side issues. Implementing robust error-handling with retries can also help mitigate these problems.
You can find more information and context in the Prefect documentation:
- Concepts/Infrastructure
Let me know if you need further assistance or details on how to implement these solutions in your specific setup.
n
Nate
03/28/2024, 4:55 PM
hi @Joe D - what version of
prefect-aws
are you on?
in 0.4.12 we added some logic to avoid registering extra task definitions
j
Joe D
03/28/2024, 5:29 PM
0.4.7
- so an upgrade might fix this?
n
Nate
03/28/2024, 5:32 PM
if you're hitting the case that we addressed here, then its likely yeah
Nate
03/28/2024, 5:32 PM
you'd have to flip on the option on your work pool
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.