Currently running into this issue: `botocore.error...
# prefect-aws
j
Currently running into this issue:
botocore.errorfactory.ClientException: An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Too many concurrent attempts to create a new revision of the specified family.
Our workflow is that we are using an
ECSTask
block with github storage.
Copy code
prefect-aws==0.1.8
prefect==2.6.8
This is mainly caused when a mother flow fires off a subflow via
run_deployment
, but can occur when trying to invoke a standalone flow
b
this was fixed with the most recent version of
prefect-aws
πŸ™ 2
off the top of my head there were some breaking changes, so make sure you read the change log before upgrading.
j
@Ben Muller currently getting the same issue with
Copy code
prefect==2.7.10
prefect-aws==0.2.4
b
Does your agent have the new versions?
j
yes, agent and server were updated to use the docker image
prefecthq/prefect:2.7.10-python3.9
, logic used within the Dockerfile:
Copy code
FROM prefecthq/prefect:2.7.10-python3.9 
RUN pip install prefect-aws==0.2.4
RUN prefect block register -m prefect_aws.ecs
b
That's very bizarre. You might have to post what your EcsTask looks like. It might be configured wrong.
j
Copy code
ecs_task_fg = ECSTask(
        aws_credentials=aws_credentials_block,
        image=docker_image,
        cpu=4096,
        memory=8192,
        stream_output=True,
        configure_cloudwatch_logs=False,
        cluster=os.getenv('ECS_CLUSTER'),
        execution_role_arn=os.getenv('FLOW_EXECUTION_ROLE'),
        task_role_arn=os.getenv('FLOW_TASK_ROLE'),
        vpc_id=os.getenv('VPC_ID'),
        env=flow_environment,
        task_definition=task_def,
    )
task_def
is read with `pyaml-env`'s
parse_config
function and is a dictionary that describes sidcars loaded with the task, it also contains logging configuration for the task container as well (we use DataDog to handle logging).
flow_environment
is a dict of envars.
docker_image
is a string of the ECR-stored image
b
Hmmm this is where my help might run to an end. Not exactly sure of the inner workings, @Anna Geller will most likely be able to help though ☺️
a
does it work without sidecars? not sure I can help with those patterns
j
Just tried without the sidecars and receiving the same error.
seems including the
task_definition=
argument was the issue. With that completely removed (not just the sidecars), we are not seeing this issue.
πŸ‘ 1
a
nice, that makes sense!
j
Made the change (removed the
task_definition=
argument), then went on vacation. When I returned, the too-many-revisions error (which we have been calling this) resurfaced while I was gone. Would love to get some insight into why this is happening.
for anyone else getting these errors, our fix was to define the task definition as part of our CodePipeline and then use that task_definition_arn when creating the
ECSTask
infrastructure block. This method also allows for the loading of sidecars.
πŸ™Œ 1
gratitude thank you 1