https://prefect.io logo
m

Mac

02/14/2022, 5:54 PM
Hello, I am trying to run an ECS task while defining the task definition as a dictionary (https://github.com/anna-geller/packaging-prefect-flows/blob/master/flows/s3_ecs_run_task_definition_as_dict.py), but I keep receiving this error:
An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Container.image should not be null or empty
I'm have defined the task as a yaml file (e.g. https://github.com/anna-geller/packaging-prefect-flows/blob/master/flows/s3_ecs_run_custom_task_definition.py) and that works fine, so I am not sure what I am doing wrong. Thanks!
k

Kevin Kho

02/14/2022, 6:33 PM
Yeah that’s a bit weird. This errors during runtime right? Is there something on the agent side like a task definition?
Actually, did you change the
name
to something other than
flow
?
m

Mac

02/14/2022, 7:08 PM
Yes the error is during runtime. I am trying to define the task definition within the task, but am going to test defining the task definition on the agent as well. I'm not sure what you mean my changing the name. Here is my run config:
Copy code
RUN_CONFIG = ECSRun(
    labels=["prod"],
    task_definition=dict(
        family="*********",
        requiresCompatibilities=["FARGATE"],
        executionRoleArn="arn:aws:iam::*******:role/**********",
        taskRoleArn="arn:aws:iam::*********:role/*****",
        networkMode="awsvpc",
        cpu=512,
        memory=1024,
        containerDefinitions=[
            dict(
                name="*********",
                image="*****.<http://dkr.ecr.us-east-1.amazonaws.com/****:latest|dkr.ecr.us-east-1.amazonaws.com/****:latest>",
                essential=True,
                logConfiguration=
                    dict(
                        logDriver="awslogs",
                        options=
                            {
                                "awslogs-group":"/ecs/***",
                                "awslogs-region":"us-east-1",
                                "awslogs-stream-prefix":"ecs",
                            },
                    ),
            
                secrets=[
                    dict(
                        valueFrom="arn:aws:secretsmanager:*****",
                        name="secret_name",
                    )
                ],
            ),
        ],
    ),
    run_task_kwargs=dict(cluster="prefectEcsCluster"),
)
When I try to run a pre-registered task definition, I get this error:
botocore.errorfactory.InvalidParameterException: An error occurred (InvalidParameterException) when calling the RunTask operation: Override for container named flow is not a container in the TaskDefinition.
I assume these errors are related
k

Kevin Kho

02/14/2022, 7:29 PM
Prefect requires the container
name
to be named
flow
for the one that actually runs the Flow
m

Mac

02/14/2022, 7:33 PM
Oh I see, that did the trick. Thanks!
k

Kevin Kho

02/14/2022, 7:36 PM
No problem!
m

Mac

02/14/2022, 8:28 PM
Hey @Kevin Kho, one additional question. Our Data Pipelines run via ECS Tasks. So that's what I am triggering above. The pipeline has run successfully, but it appears that flow is not recognizing it as successful. Instead, a Lazarus process is retrying the task. Is there something special that I need to return in the task so that prefect moves on to the next step?
k

Kevin Kho

02/14/2022, 8:31 PM
You shouldn’t have to. Do all the tasks complete successfully?
m

Mac

02/14/2022, 8:34 PM
Yes they do
k

Kevin Kho

02/14/2022, 8:37 PM
That’s weird. Do you have an unclosed connection by chance?
m

Mac

02/14/2022, 9:06 PM
I don't believe so, all the tasks were successfully stopped
k

Kevin Kho

02/14/2022, 9:06 PM
Do you have a state handler at the end of the Flow? Is this happening all the time?
m

Mac

02/14/2022, 9:18 PM
Yeah, happening each time - I have not added a state handler yet. I am just testing out some basic pieces first
k

Kevin Kho

02/14/2022, 9:20 PM
Can you DM me your Flow so I can take a look? First time I’ve seen this behavior on ECS