Hello, has anyone been able to workaround the `Fai...
# prefect-kubernetes
d
Hello, has anyone been able to workaround the
Failed to establish a new connection: [Errno 99] Cannot assign requested address
issue with prefect workers? Currently we have to periodically restart our workers in order to avoid the work pool getting stuck on that error. Unfortunately that causes other issues as Prefect is apparently not able to gracefully handle worker restarts while it has flows in "Pending"/"Cancelling"/"Running" states. These flows will stay in those states forever as their worker doesn't exist anymore and they still count towards the work pool concurrency limit so everything just runs to a screeching halt. I have to run a manual script to force delete/reschedule these stuck flows and release the concurrency limit slots. How are people using Prefect work pools without the need of constant human supervision and manual scripts? Related issues: https://github.com/PrefectHQ/prefect-kubernetes/issues/102 https://discourse.prefect.io/t/flow-stuck-in-pending-mode/3406
cc @Sahil Rangwala as discussed on the support call