Nikhil Jain
02/07/2023, 6:15 PMAn error occurred (ClientException) when calling the RegisterTaskDefinition operation: Too many concurrent attempts to create a new revision of the specified family.
Here’s my setup: prefect==2.7.0
, prefect-aws==2.1.0
. ECS block definition:
ecs_task_block = ECSTask(
task_definition_arn=task_def_arn,
cluster=f'{env}-prefect-cluster',
image=f'123456789.dkr.ecr.{regions[env]}.<http://amazonaws.com/{image_region[env]}-stride-enrollment-flows-ecr:{ecr_image_tag}|amazonaws.com/{image_region[env]}-stride-enrollment-flows-ecr:{ecr_image_tag}>',
vpc_id=vpcs[env],
task_customizations=[
{
"op": "add",
"path": "/networkConfiguration",
"value": {
"awsvpcConfiguration": {
"subnets" : subnets[env],
"securityGroups" : security_groups[env],
"assignPublicIp" : "DISABLED"
}
}
}
],
configure_cloudwatch_logs=True,
stream_output=True,
)
ecs_task_block.save(f'{env}-ecs-block', overwrite=True)
I can see that prefect is still creating new task definitions; one of the things it is overriding on the task def is aws-logs-prefix
to be the flow-run-id. Is there a different way to be able to access logs in cloudwatch without setting a different name?configure_cloudwatch_logs
to False
. I am wondering if this will effectively stop logging flow run logs to cloudwatch? Or will they still be available but with a different name?
2. I can try to use different families for each flow so that the request throttling applies separately for different runs instead of collectively. However, I am wondering if this line of thinking is correct because AWS Request Throttling document seems to suggest that the throttling is per account per region. So having multiple families may not help?