Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

image.png

Hello again! I've got a couple of tasks set up to run with ECSRun. I have an error in my container (which I'm aware of). However, it looks like Prefect isn't getting the feedback that the flow even started. Prefect shows it getting stuck in the submitted state. However, ECS has already killed the task. Any thoughts on how to get this feedback to Prefect?

Hey <@U027K8BSQ7J>, was the Flow able to start and the error was during running the Flow?

hey Kevin, it doesn't look like any of the DBT tasks started. I threw a basic ShellTask (just the ls command) in there for the first task, so that one shouldn't fail. However, I'm guessing since Prefect didn't report back on that task that it didn't make it that far?

Does this basically mean it is failing before it gets to any of the prefect code?

I suspect so because it looks like the logger (and connection) wasn’t established and something failed earlier in the container. Do you see anything on the CloudWatch logs side?

Maybe you can test that container by using `DockerRun` and a Docker agent first?