Darragh
06/03/2020, 8:22 PMflow.environment = FargateTaskEnvironment(
launch_type="FARGATE",
region="eu-west-1",
cpu="256",
memory="512",
networkConfiguration={
"awsvpcConfiguration": {
"assignPublicIp": "ENABLED",
"subnets": ["subnet-X"],
"securityGroups": ["sg-Y"],
}
},
family="my_flow",
taskRoleArn="arn:aws:iam::X:role/CommonSuperRole",
executionRoleArn="arn:aws:iam::X:role/CommonSuperRole",
containerDefinitions={
"name": "my-flow",
"image": "my-flow",
"command": [],
"environment": [],
"essential": True,
}
)
I keep getting this error:
An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Fargate requires task definition to have execution role ARN to support log driver awslogs
So my questions :
• I’m using the same Uber Role for both the taskRoleArn and the executionRoleArn, probably not best practice but should work?
• I’ve thrown every possible log and cloudwatch related policy/permission at it that I can think of, but nothing is taking. I can provide a dump of the permissions if need be?
Any help massively appreciated, fairly stumped on it. Is there any way to get more debug info out of it?Joe Schmid
06/03/2020, 8:27 PMecsTaskExecutionRole
It looks like yours are set to the same role. See this in AWS docs: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_execution_IAM_role.htmlecsTaskExecutionRole
) that has both of those words in the name!?)Darragh
06/03/2020, 8:34 PMThe task execution role is supported by Amazon ECS container agent version 1.16.0 and later
Joe Schmid
06/03/2020, 8:37 PMecsTaskExecutionRole
2. Update your FargateTaskEnvironment to use the ecsTaskExecutionRole
for execution role and your own role for task role, like this:
task_role_arn="arn:aws:iam::<your-aws-account-number>:role/<your-aws-iam-role-name>",
execution_role_arn="arn:aws:iam::<your-aws-account-number>:role/ecsTaskExecutionRole",
Darragh
06/03/2020, 8:45 PMflow.environment = FargateTaskEnvironment(
launch_type="FARGATE",
region="eu-west-1",
cpu="256",
memory="512",
networkConfiguration={
"awsvpcConfiguration": {
"assignPublicIp": "ENABLED",
"subnets": ["subnet-RANDOM"],
"securityGroups": ["sg-RANDOM"],
}
},
family="my_flow",
taskRoleArn="arn:aws:iam::RANDOM:role/Orchestration-PrefectRole1E6EFC48-1J7FK5V772G96",
executionRoleArn="arn:aws:iam::RANDOM:role/ecsTaskExecutionRole",
containerDefinitions={
"name": "batch-universe",
"image": "batch-universe",
"command": [],
"environment": [],
"essential": True,
}
)
Joe Schmid
06/03/2020, 9:07 PM"log driver awslogs"
-- we specifically add that to our containerDefinitions but you don't have that in your config. Here's ours:
containerDefinitions=[
{
"command": [],
"environment": [
{"name": "PREFECT__LOGGING__LOG_TO_CLOUD", "value": "true"},
{"name": "AWS_DEFAULT_REGION", "value": REGION_NAME},
],
"essential": True,
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "blah-blah",
"awslogs-region": "our region",
"awslogs-stream-prefix": "blah blah",
},
},
}
],
Darragh
06/03/2020, 9:12 PMJoe Schmid
06/03/2020, 9:46 PMecsTaskExecutionRole
for execution role and your CommonSuperRole
for task role with the agent. (Ignore me if your Fargate Agent is already working fine.)Darragh
06/03/2020, 9:50 PMJoe Schmid
06/03/2020, 9:50 PMDarragh
06/03/2020, 9:52 PMJoe Schmid
06/03/2020, 9:52 PMDarragh
06/03/2020, 9:58 PMAn error occurred (ClusterNotFoundException) when calling the RunTask operation: Cluster not found
Joe Schmid
06/04/2020, 12:16 AMSo i might not actually need the fargate task envrionmwnt at all, is that right?That’s correct. There’s certainly no requirement to use it if you’re using the Fargate Agent.
a new error!!
That’s definitely progress! Since the error is in RunTask, your Fargate Agent was able to successfully register a task definition. (And is now trying to run it.) Since yourAn error occurred (ClusterNotFoundException) when calling the RunTask operation: Cluster not found
launch_type
is FARGATE
I think it should use the default cluster in ECS if you don’t specify one, but maybe try specifying a cluster ARN for your Fargate Agent. You also mentioned “Added ARNs to both the flow and the agent” — if the Flow is still using FargateTaskEnvironment, I would switch that to RemoteEnvironment().Leonard Marcq
09/12/2020, 9:12 PMAn error occurred (ClientException) when calling the RegisterTaskDefinition operation: Fargate requires task definition to have execution role ARN to support log driver awslogs
error message and might be searching for help in the channel later. Make sure that you spelled taskRoleArn
and executionRoleArn
in camelCase and not in snake_case. Just spent a good billion hours looking for the source of the problem everywhere else but in my spelling lolMaikel Penz
10/13/2020, 8:41 AMCluster not found
error? I am facing the sameDarragh
10/13/2020, 9:16 AMMaikel Penz
10/13/2020, 7:05 PMAn error occurred (ClusterNotFoundException) when calling the RunTask operation: Cluster not found