https://prefect.io logo
Title
c

Carlo

11/17/2022, 12:23 AM
Hello. We were originally using
ECSTask
with the
task_definition
arg. Today we tried transitioning to
task_definition_arn
However, it's still trying to register a task definition every time the agent picks up a flow to run. Details in thread...
1
Below is how we create the task. The task_definition_arn has an image, but if we don't pass the image tag explicitly to the ECSBlock, the default prefect image gets used. I'm not sure if it's that or the task_customizations that trigger a new revision to get created for the family of the task def. We were trying to transition away from creating new revisions so that we can remove our agent hot fix that included the flow_run_id in the family name to avoid aws rate limits. Prefect version 2.6.4
ecs = ECSTask(
    stream_output=False,
    configure_cloudwatch_logs=True,
    cluster="my-cluster",
    image=CONTAINER_IMAGE_PYTHON,
    vpc_id=VPC_BY_ENV[env],
    task_customizations=ENV_OVERRIDES[env_override_key],
    task_definition_arn=task_arn,
    task_start_timeout_seconds=60*6
)
m

Mason Menges

11/17/2022, 12:33 AM
Hey @Carlo you'd actually need to set the image to None, this thread in the #prefect-aws channel covered a similar thing https://prefect-community.slack.com/archives/C048K0MGHNK/p1668546666300229?thread_ts=1668546666.300229&cid=C048K0MGHNK
c

Carlo

11/17/2022, 12:36 AM
Thanks, will try this
Thanks this helped. As an FYI, if the parameter
configure_cloudwatch_logs
is True, it creates a new task def revision to update the
awslogs-stream-prefix
with the flow run name, eg mighty-adder