Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hey all, we are running prefect server v3.0 on AWS EKS and we have some issues related to how Karpenter reshuffles nodes which leads to the prefect server and/or prefect flows in prefect deployment runs on pods to be restarted in new pods.

Related to this, we were wondering if prefect server is able to shut down gracefully?

Any tips on how to make sure that prefect server runs stable and we don't get the following error would be highly appreciated: `Job reached backoff limit`

<@U024EMRLBCH> is a fellow Karpenter guru

I would suggest adding the do-not-disrupt annotation
```            "annotations": {
              "<http://karpenter.sh/do-not-disrupt|karpenter.sh/do-not-disrupt>": "true"
            }```
this will prevent shutdown

Well this would only help with node group consolidation - if you use spot instance, they can be removed at any time