Thread
#prefect-community
    s

    Sridhar

    11 months ago
    Hi folks, I used (https://www.lejimmy.com/distributed-data-pipelines-with-aws-ecs-fargate-and-prefect-cloud/) this tutorial to get started with prefect. I've followed every step exactly as mentioned but when I start the flow on prefect cloud it is stuck at Submitted for execution: Task arn:aws:ecs:ap-southeast-2:037961805145:task/prefect-fargate-cluster/a4bced3ed09f43ef92177c492cdc0cf0 and fails after 3 attempts. here's the agent code:
    from prefect.agent.ecs.agent import ECSAgent
    AGENT = ECSAgent(cluster="prefect-fargate-cluster", labels=['ecs', 'dev'])
    AGENT.start()
    flow.py code:
    import prefect
    from prefect.storage import Docker
    from prefect.run_configs import ECSRun
    from prefect import task, Flow, Parameter
    STORAGE = Docker(registry_url='<http://037961805145.dkr.ecr.ap-southeast-2.amazonaws.com/|037961805145.dkr.ecr.ap-southeast-2.amazonaws.com/>',
                       
    image_name='prefect-etl-flow',
                       
    image_tag='latest',
                     
    dockerfile='Dockerfile.txt')
     
    RUN_CONFIG = ECSRun(run_task_kwargs={'cluster': 'prefect-fargate-cluster'},
                        
    execution_role_arn='arn:aws:iam::037961805145:role/prefect-ecs',
                        
    labels=['ecs', 'dev'])
    @task
    def testfunc():
        
    print("Hello prefect!")
    with Flow('prefect-etl-flow', storage=STORAGE, run_config=RUN_CONFIG) as flow:
        
    testfunc()
    flow.register('emmi-etl')
    Really appreciate the help. Thanks in advance.
    Kevin Kho

    Kevin Kho

    11 months ago
    Hey @Sridhar, hard to tell immediately what happened. Are there Task logs on ECS that you can check?
    s

    Sridhar

    11 months ago
    Hey @Kevin Kho, I haven't set up logs yet. I noticed that the task was inactive on ecs. But weirdly the flow worked the next day without having to change anything. Any idea where I should be looking at? Just wanna know why it didn't work at first and then it fixed itself??
    Kevin Kho

    Kevin Kho

    11 months ago
    I don't think you need to set up logs? Sometimes you'll find in the task tab. This sounds like maybe you just weren't able to get compute properly on the first try? Let's see if it succeeds/fails again. Or maybe your permissions changed