I m running Prefect 2 0 Flows over an AWS ECS stack Fargate Prefect Community #ask-community

I’m running Prefect 2.0 Flows over an AWS ECS stac...

Robert Banick

06/20/2023, 5:31 PM

I’m running Prefect 2.0 Flows over an AWS ECS stack (Fargate and EC2). The relevant infrastructure blocks are created by specifying a

task_definition_arn

corresponding to the desired infrastructure. Occasionally our flows fail with the “too many concurrent attempts”

ClientException

mentioned in this thread. https://discourse.prefect.io/t/i-am-using-ecsagent-and-sometimes-i-get-an-error-that-sa[…]empts-to-create-a-new-revision-of-the-specified-family/100. I have two questions: 1. Is it really necessary to register and de-register a task definition for every flow run? Can Prefect not just use the stable task definition? 2. If yes, is this agent-based solution for Prefect 1 still the recommended solution for Prefect 2? https://docs-v1.prefect.io/orchestration/agents/ecs.html#throttling-errors-on-flow-submission. That is, will the Prefect Agent recognize and use the

AWS_RETRY_MODD

and

AWS_MAX_ATTEMPTS

environment parameters?

Robert Banick

06/20/2023, 5:39 PM

Our ECS Task blocks look like so. Are the

task_customizations

or other specifications forcing Prefect to register anew each time?

Copy code

ECSTask(
    aws_credentials=AwsCredentials.load("creds"),
    task_definition_arn="arn:aws:ecs:us-east-1:123456789:task-definition/specific-task:1",
    vpc_id="123456789",
    stream_output=True,
    configure_cloudwatch_logs=True,
    cluster="OurCluster",
    command=["a bash command"],
    task_start_timeout_seconds=300,  # need to expand beyond default 120 to accommodate bash command
    task_customizations=[
        {
            "op": "add",
            "path": "/networkConfiguration/awsvpcConfiguration/subnets",
            "value": [
                "subnet-1",
                "subnet-2",
                "subnet-3",
            ],
        },
        {
            "op": "add",
            "path": "/networkConfiguration/awsvpcConfiguration/securityGroups",
            "value": ["abcd"],
        },
    ],
)

2 Views

Open in Slack

Previous Next