Hey all, I am running a prefect agent from Command Prompt on my company's on-prem server from a remote session. Is there a way to programmatically re-kick the agent from that on-prem remote machine? For example, I would like to create an automation that runs the command "prefect agent start" on the remote session when the work pool enters "unhealthy" status for more than 10 minutes. How would I go about doing that?

Hey Grant, thanks for that question - I can think of some tape and glue methods here, none of which feel great. If your agent needs to be restarted frequently, that sounds like an upstream problem we should actually look into. Under what circumstances do you find the agent process having died or in need of a restart?

Yeah, a couple of times now when I see the work pool gets stuck in "unhealthy" status and I check the cmd in my remote session, the agent has stopped and the error message in cmd says "an exception has occurred". My logs don't capture agent issues so I don't have the full error message, but I notice that some errors occur in the output of the cmd while running flows leading up until the time of the exception and ending of the agent. I will follow up with the full error message the next time this happens.

You should be able to safely ignore that warning. Will is the person for automation questions, so you’re in good hands.

yes please do capture the error message if possible. Even better, check out using the process worker, the next generation of the agent, which which send worker logs to the server.

The process worker is coming up in about 10 minutes in PACC, @Grant. 🙂

@Will Raphaelson wanted to follow up with the error log from this weekend where my agent went down (see attached). I have tried to implement a process worker, but had issues getting my flow runs to run successfully using a worker rather than an agent. Do you have any insight? See that thread: https://prefect-community.slack.com/archives/CM28LL405/p1688752320734569

Thanks grant - is the bonus results work pools the agent type or process type? you’ll have to create a work pool of process type to start a worker that can pick up work from it. Not sure why you’re getting that 408 though, let me ask some folks internally.

Thanks Will. Yep I created a Process type work pool as mentioned in that thread, and what was weird was that the flow kicked off successfully but kept getting hung up on the final task. Would it be a matter of resource allocation?

i dont think so but ive asked for some support from engineering on this. one question - are you on cloud or open source?

I am on prefect cloud 2!

trying to work through what could be going on here. do you mean resource of the execution environment? Can i ask where you are running these things? Thanks.

@Will Raphaelson I am running the worker and agent on an on-prem server from command prompt - yes I was referring to the resources of the execution environment, sorry for the lack of clarity. I will submit an issue as soon as I have a chance, thank you!

Okay. And one way to test is this is a resource issue could be to either up the size of the box you’re running on, or profile the code.