Hi folks, I used (https://www.lejimmy.com/distributed-data-pipelines-with-aws-ecs-fargate-and-prefect-cloud/) this tutorial to get started with prefect. I've followed every step exactly as mentioned but when I start the flow on prefect cloud it is stuck at Submitted for execution: Task arnawsecsap southeast 2037961805145:task/prefect-fargate-cluster/a4bced3ed09f43ef92177c492cdc0cf0 and fails after 3 attempts. here's the agent code:

from prefect.agent.ecs.agent import ECSAgent

AGENT = ECSAgent(cluster="prefect-fargate-cluster", labels=['ecs', 'dev'])

AGENT.start()

flow.py code:

import prefect

from prefect.storage import Docker

from prefect.run_configs import ECSRun

from prefect import task, Flow, Parameter

STORAGE = Docker(registry_url='<http://037961805145.dkr.ecr.ap-southeast-2.amazonaws.com/|037961805145.dkr.ecr.ap-southeast-2.amazonaws.com/>',

                   

image_name='prefect-etl-flow',

                   

image_tag='latest',

                 

dockerfile='Dockerfile.txt')

 

RUN_CONFIG = ECSRun(run_task_kwargs={'cluster': 'prefect-fargate-cluster'},

                    

execution_role_arn='arn:aws:iam::037961805145:role/prefect-ecs',

                    

labels=['ecs', 'dev'])

@task

def testfunc():

    

print("Hello prefect!")

with Flow('prefect-etl-flow', storage=STORAGE, run_config=RUN_CONFIG) as flow:

    

testfunc()

flow.register('emmi-etl')

Really appreciate the help. Thanks in advance.

Hey @Sridhar, hard to tell immediately what happened. Are there Task logs on ECS that you can check?

Hey @Kevin Kho, I haven't set up logs yet. I noticed that the task was inactive on ecs. But weirdly the flow worked the next day without having to change anything. Any idea where I should be looking at? Just wanna know why it didn't work at first and then it fixed itself??

I don't think you need to set up logs? Sometimes you'll find in the task tab. This sounds like maybe you just weren't able to get compute properly on the first try? Let's see if it succeeds/fails again. Or maybe your permissions changed