hello community, hello prefect team, are there any best practices about updating agents or workers with zero downtime and data loss? we are using a custom infrastructure block (refactoring to custom worker type now) that is in active development and therefore requires us to restart the agents or workers relatively often to update to a new version. flow runs are tracked by an agent after they are started, so restarting an agent while a flow is still running will lose track of it in case of infra crashes and since there is no functionality to drain agents, we cannot smoothly rollover to another agent. is there any process that i am not aware of that would enable spinning up a new agent, direct all new flows to this agent and shutdown the old agent as soon as its last flow terminates? thanks in advance

Hey @Justin Trautmann! You can gracefully stop a worker or agent with a SIGINT, which will tell the worker or agent to stop queueing new flow runs and finish their current runs. This should allow you to transition agent or worker versions without downtime by starting new workers and sending a SIGINT to the old agents or workers.

oh that is good news. thank you