https://prefect.io logo
Title
j

jack

11/22/2022, 9:27 PM
Is it safe to stop the prefect agent in order to update the packages installed in the virtualenv? Or would that cause any flow-runs-in-progress to be killed/restarted? Asking for both prefect 1.x and 2.x, as we are using both.
When running
pipenv sync
with the agent still running (on Windows), the command fails with error:
ERROR: Could not install packages due to an OSError: [WinError 32] The process cannot access the file because it is being used by another process: '[REDACTED]\\prefect.exe'
k

Kevin Wang

11/23/2022, 1:12 AM
Hey, for local infrastructure, those runs are 'lost'.. they get stuck on 'Running' forever, and no agent will pick those runs up anymore. I've had to manually clean up those runs, by finding those Running for >> long hours, and setting their state to Failed/Crashed. (but if your jobs are executing elsewhere, you may be fine?)
Is there a way to control an agent to 'draining', where it stops reading from queues? Then any local flow runs still running will complete, and then the agent shuts down? This would help us in cases of local infra; instead of having your flow runs die when the agent is restarted, we would: 1) Drain the current agent 2) Startup a new agent
1
z

Zanie

11/23/2022, 3:44 PM
I think we want to adjust the agent’s subprocesses to be daemons so the agent can exit and the flow run subprocesses will continue running
:upvote: 1
:gratitude-thank-you: 1
j

jack

11/23/2022, 4:17 PM
Is there a way to query the agent to see if any flow runs are in flight? (To check whether safe to stop the agent to update virtualenv)
z

Zanie

11/23/2022, 4:21 PM
Not right now, we’re planning some work in the future that would address this case though.
k

Kevin Wang

11/23/2022, 4:59 PM
we want to adjust the agent’s subprocesses to be daemons so the agent can exit and the flow run subprocesses will continue running
i see, thanks for responding. I now get a better sense for what's happening in #7239, which you linked on Github 🙂
🙌 1