Thread
#prefect-community
    d

    David

    1 year ago
    Hi all 🙂 I am encountering an issue regards the new ECS agents and I will be glad for some advice, we are executing many flows nightly. For the last half year used the Fargate agents, which worked pretty fine, we tried to move to the new agent - ECS agent but now after deploying new flow and running the batches we receive -
    An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Too many concurrent attempts to create a new revision of the specified family.
    The ECS agent logs (verbose) isn’t very helpful and I can’t debug the error as it happens when the system at its peak. And I have couple of questions -1. Why does the flow register itself on AWS only when its running and not when it was registered? 2. Can I ask prefect to just dockerize my flow instead of register a new task definition (because nothing changed in the task definition), I saw the parameter of
    idempotency_key
    but I don’t understand if this is helpful in my situation 3. Should I call
    register
    if my docker is the only thing changed? or should I call
    flow,storage.build
    instead? I can provide code if needed, Thank you in advance
    Kevin Kho

    Kevin Kho

    1 year ago
    Hi @David, I believe there was a recent PR that will help you with this here . This is because of AWS’s low rate limit on the RegisterTaskDefinition endpoint. You can try
    prefect agent ecs start --task-definition-add-uuid true
    d

    David

    1 year ago
    Hi, thank you that looks like a solution for us. Which version of prefect supports it?
    Looks like this PR wasnt merge into a release, how should I use it? should I wait for a release?
    Kevin Kho

    Kevin Kho

    1 year ago
    I think the issue continued on and there is no solution yet. If you really need it immediately, I guess the approach would be to clone the prefect repo, edit the code as he did, and use that.