Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hi! Switching my flows from running on a LocalAgent hosted on an EC2 instance to an ECSAgent also hosted on an EC2 instance. I've got a small flow that runs on an hourly basis up and running that executes correctly, but once every few flows it fails with the comment "Flow run 9f8a7552-90f8-49d2-8091-31db5e7ed1f3 has a `run_config` of type `ECSRun`, only `LocalRun` is supported", the next scheduled flows will continue to fail until I restart the agent

On the agent side, I see `Deploying flow run 9f8a7552-90f8-49d2-8091-31db5e7ed1f3 to execution environment...`, but it never says that the deployment has been completed

basically it seems like the agent is getting stuck for some reason and then hanging, and so only the LocalAgent is available to pick up new flows

Looking in the UI, it looks like both agents are green and healthy. Thoughts on what could be going on here?

Hey <@U022KGKE4TG>, I think you need `labels` to avoid this with certainty. There is no guarantee just by using the ECSRun that an ECS Agent will pick it up. So it just seems that both the Local Agent and ECS Agent are polling and the Local Agent happens to pick it up. I think there is a race condition happening here with two agents that can pick up the flow, which is why you see that log. Either way, I would expect adding labels to the flow and agent would fix this.

I assume you’re familiar with labels, but attaching <https://docs.prefect.io/orchestration/flow_config/run_configs.html#labels|docs> for easy access

Ah yeah, so right now I have the flow labeled "prod", the ECS agent labeled "prod", and the LocalAgent labeled "prod", "&lt;instance address&gt;". So since the labels of both agents are a superset of the labels on the flow, both try to pick it up?

Yes that’s right. Flow labels just need to be a subset of Agent labels for the Agent to be able to pick it up

got it got it, i think i was assuming that the run_config type would also play a role. Easy fix, thank you!