https://prefect.io logo
Title
s

Samuel Bunce

02/15/2023, 8:40 AM
Hi all, a quick one about running prefect agents on windows, with enterprise cloud server. I am facing an issue where after long-running tasks complete the agents just stop. The last flows exit cleanly, and the next message I get is just
Agent stopped!
- is there any reason why agents would just be stopping? I thought they were meant to run indefinitely. I do not know where to access any logging to see why the agent was stopped.
n

nick vazquez

02/15/2023, 6:12 PM
I ran into https://stackoverflow.com/a/30517482 for general python scripts seeming to have frozen until interaction and changing the setting stopped that, but then you cant copy paste from the terminal... whomever thought that was a good idea....
Let me know if that turned out to be the issue!
s

Samuel Bunce

02/16/2023, 7:59 AM
this is really helpful thank you! The plot has thickened a bit since - something is going on which is quite worrying in a flow that uses
from multiprocessing import Manager, Process
in short, I believe this might be interfering with prefect. When I have deployed 2 agents on the same box in different queues, I have seen logs from the flow which uses multiprocessing present in an agent that is registered on a completely different work queue! This can't be expected behaviour! As a further note to my original question I believe running
manager.shutdown()
within my flow might be having some undesirable effect on the prefect agents, but I have not yet tested this. In the meantime I have disabled the flow using
multiprocessing
t

Tim-Oliver

02/16/2023, 8:21 AM
I get
Agent stopped
as well with the following error:
Failed the last 3 attempts.  Please check your environment and configuration.
Examples of recent errors:

Traceback (most recent call last):
  File "miniconda3/envs/prefect2-test/lib/python3.9/site-packages/h2/connection.py", line 224, in process_input
    func, target_state = self._transitions[(self.state, input_)]
KeyError: (<ConnectionState.CLOSED: 3>, <ConnectionInputs.SEND_HEADERS: 0>)
This error shows up after the agent was running for more than 24h. Would you say this is similar to your issue or completely different?
s

Samuel Bunce

02/16/2023, 8:48 AM
yes, that's exactly the error
but sometimes I don't get the traceback
t

Tim-Oliver

02/17/2023, 7:48 AM
My agent was dead again this morning. This is getting a bit annoying.
s

Samuel Bunce

02/17/2023, 8:48 AM
same here
this is rendering prefect unusable on windows - i think it is a recent breaking change
t

Tim-Oliver

02/17/2023, 10:19 AM
Now it just died again after < 3h on a linux machine.
I have now downgraded to 2.7.9, since these seemed to work more stable.
s

Samuel Bunce

02/20/2023, 8:59 AM
thanks for submitting this - surprised more people aren't encountering this