https://prefect.io logo
Title
a

Ashley Felber

11/30/2022, 7:06 PM
Hello, I have an agent set up on ECS and built a deployment using the ECStask block to run the flow as a Fargate task. I tried to run the deployment and the flow run failed. I have no information as to why though. The logs are completely empty, it says "This run didn't generate Logs". There is also no logs in AWS for the task I was intending to run so it seems like it failed before the task even started.
cc: @Taylor Curran
m

Mike Grabbe

11/30/2022, 7:36 PM
did you create the ECS agent like in this example repo? https://github.com/anna-geller/dataflow-ops/tree/main/infrastructure
:gratitude-thank-you: 1
:blob-attention-gif: 1
I found that I needed to add a couple extra IAM permissions to get it work for me
a

Ashley Felber

11/30/2022, 7:42 PM
No I created according to this. https://github.com/PrefectHQ/prefect-recipes/tree/main/devops/infrastructure-as-code/aws/cli-prefect2-ecs-agent. Did you have the same issue that there were no logs when you tried to run?
m

Mike Grabbe

11/30/2022, 7:47 PM
I think those cli commands will only get you as far as having a prefect agent process running in ECS Fargate. It would be able to run in-process tasks (same container as the agent), but to get it to spin up other ECS fargate containers to run tasks, a bit more is required
šŸ™Œ 1
for example, the task role needs permissions to create task definitions and run ecs tasks. You can find the exact iam permissions in that dataflow-ops repo.
a

Ashley Felber

11/30/2022, 8:01 PM
Yeah it already has the permission to run ecs tasks and the task definition is created - I am running an existing task definition. So I would think the ability to run an ecs task is the only permission needed.
m

Mike Grabbe

11/30/2022, 8:06 PM
hmm two questions: ā€¢ does your ECSTask block specify an image? ā€¢ did you include
ecs:DescribeTasks
as well?
a

Ashley Felber

11/30/2022, 8:06 PM
Yes to the first, will check on the second.
m

Mike Grabbe

11/30/2022, 8:06 PM
ok, try setting the image to None
image conflicts with the task definition arn... when an image is set, it tries to create a new task definition no matter what
a

Ashley Felber

11/30/2022, 8:08 PM
Got it, I actually tried to remove the image before. I edit, remove, save. But when I refresh the page, the image is still there. Just tried again.
m

Mike Grabbe

11/30/2022, 8:08 PM
oh weird. Were talking about the ECSTask block, right?
a

Ashley Felber

11/30/2022, 8:09 PM
Yup correct
I can take a video of what's happening if that's helpful
m

Mike Grabbe

11/30/2022, 8:15 PM
Well if its some kind of UI bug, Prefect will need to look into that. But I can show you how I set up our own ECSTask block:
from prefect_aws.ecs import ECSTask

ecs = ECSTask(
    name="Run ecs task in fargate using pre-existing task definition",
    image=None,
    task_definition_arn=f"arn:aws:ecs:us-east-1:00000000:task-definition/my_task_defintion_name:1",
    cpu=1024,
    memory=2048,
    stream_output=True,
    launch_type="FARGATE",
    configure_cloudwatch_logs=True,
    vpc_id="vpc-00000000",
    cluster="prefect",
    task_start_timeout_seconds=180,
    execution_role_arn="arn:aws:iam::0000000000:role/dataflowops_ecs_execution_role",
    task_role_arn="arn:aws:iam::000000000:role/dataflowops_ecs_task_role",
    task_customizations=[
        {
            "op": "add",
            "path": "/networkConfiguration/awsvpcConfiguration/subnets",
            "value": ["subnet-000000000", "subnet-00000000"],
        },
    ],
)
ecs.save("ecs-fargate", overwrite=True)
šŸ™Œ 1
šŸ‘ 2
:gratitude-thank-you: 1