Prefect server stopped working and not responding tried `pre

Question

Prefect server stopped working and not responding tried `prefect server stop` and then `prefect server start detach use volume` but it get stuck ```$ prefect server start detach use volume Pulling postgres done Pulling hasura done Pulling graphql done Pulling towel done Pulling apollo done Pulling ui done Creating network prefect server with the default driver temp data processing postgres 1 is up to date Creating temp data processing hasura 1 done temp data processing graphql 1 is up to date Creating temp data processing towel 1 done Creating temp data processing apollo 1 done Creating temp data processing ui 1 done``` does not detach on on the UI I get

Kevin Kho · Answer

Did you upgrade Prefect by chance Asking because 0 15 5 has different behavior

YD · Answer

no

YD · Answer

does it do it automatically in some way

Kevin Kho · Answer

No the opposite There were breaking changes there for some users in terms of accessing the API I think your API may have failed to boot up Also check the memory of the machine

YD · Answer

stopping and trying to restart again I get ```$ prefect server start detach use volume Pulling postgres done Pulling hasura done Pulling graphql done Pulling towel done Pulling apollo done Pulling ui done Creating network prefect server with the default driver Starting temp data processing postgres 1 error ERROR for temp data processing postgres 1 Cannot start service postgres network lt some code gt not found ERROR for postgres Cannot start service postgres network lt some code gt not found ERROR Encountered errors while bringing up the project ```

Kevin Kho · Answer

Will ask the team for ideas

YD · Answer

this is the production server BTW

YD · Answer

I also have this in the crontab ```0 8 docker exec it temp data processing postgres 1 sh c psql PGPASSWORD= lt PW gt U prefect d prefect server c DELETE FROM log WHERE timestamp lt NOW INTERVAL 14 days ```

Kevin Kho · Answer

would you be able to check the postgres container for logs

Zanie · Answer

Is ` lt some code gt ` the literal output from docker compose

YD · Answer

I stopped then restarted without ` use volume` is started fine then stopped again Cleaned some data from the computer and started with ` use volume` now it appears to be OK

YD · Answer

when doing `df h` I see

YD · Answer

looks like it is using 34GB is this a Prefect process does 34GB make sense

YD · Answer

I reduced the days in the log to only 5

YD · Answer

still not work well the agent having problem to refresh connection

YD · Answer

the `Last queried` does not appear to be resetting

YD · Answer

the agent `Last queried` keep getting larger and then the flows get heartbeat error

Kevin Kho · Answer

are you using a DaskExecutor This is when you run a flow right Is the flow memory intensive Do you have a very large map operation 10000 tasks

YD · Answer

task running on the same time are in the order of several 10th

YD · Answer

maybe even less

YD · Answer

is it expected to take 34GB

Kevin Kho · Answer

34GB for a flow run It s hard to say without knowing the the flow is doing Are you working with DataFrames Or do you have a large mapped task

YD · Answer

not on the VM that runs prefect

YD · Answer

all the jobs are running on a different machine

YD · Answer

Prefect imports the functions and runs them or runs `os system cmd `

Zach Angell · Answer

Hi < YD> `overlay` is not a Prefect process I m not sure how much space it should be using There is a long thread here about cleaning up identifying contents of ` var lib docker overlay` that might be helpful Happy to help debug a bit but it s hard without more detail