Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Running into a problem with a flow whose latest run gets stuck in the "Running" state if a deployment happens to coincide with the run.
• We are using ECS to run the `prefect-agent` task that registers and runs the flows
• Whenever we make an update through our CI pipeline, a new ECS task is spun up and the old task gets deprovisioned
• Even when the old task is marked as "Stopped" in AWS (there is no hardware now running the old `prefect-agent`), the old Flow Run still says "Running" in the Prefect UI and hangs. Given our concurrency flags, it blocks all scheduled flow runs until it is manually canceled
• Questions:
    ◦ Is there a way for Prefect to detect that the old `prefect-agent` is now stopped and automatically force cancel the flow run?
    ◦ Should we be managing our deployment process differently?

Hey <@U0224D0KLJV>!! Have you looked into the Zombie Killer process (I assume this is for 1.0)? Might be worth configuring heartbeats so Prefect can detect when the ECS task has been deprovisioned: <https://docs-v1.prefect.io/orchestration/concepts/services.html#zombie-killer>