Hey all, happy new year! I have an odd failure I'm having trouble understanding. I have a flow with a single parameter input that runs on a Fargate cluster with a Dask executor. With some parameters it runs prefectly 😉 While with others, I receive the following error:

botocore.errorfactory.InvalidParameterException: An error occurred (InvalidParameterException) when calling the RunTask operation: TaskDefinition is inactive

after this:

Creating a new Dask cluster with `dask_cloudprovider.aws.ecs.FargateCluster`...

In the run config, both when it succeeds and when it fails the task_definition is null, and all the other settings are the same, has anybody encountered a similar issue and has any idea what could go wrong?

Happy New Year! It looks like a task definition revision issue - botocore doesn’t know which task definition to use, or claims that the one you implicitly specified is inactive. Can you try passing your task definition and revision explicitly? One example is below - more are here in the docstring:

flow.run_config = ECSRun(
    labels=["prod"],
    task_definition_arn="prefectFlow:1",
    # or using a full ARN:
    # task_definition_arn="arn:aws:ecs:us-east-1:XXX:task-definition/prefectFlow:1"
    run_task_kwargs=dict(cluster="prefectEcsCluster"),
)

I'm using a custom image, so I'm getting an error

ValueError: Cannot provide both `task_definition_arn` and `image`, since an ECS task's `image` can only be configured as part of the task definition

in that case you would need to set the image only in the task definition (omit image on the run config)