https://prefect.io logo
#prefect-server
Title
# prefect-server
m

Martin Felder

10/13/2021, 4:31 PM
Hi, we recently upgraded to Prefect 0.15.6, and now I'm getting these weird errors
apollo_1   | PayloadTooLargeError: request entity too large
apollo_1   |     at readStream (/apollo/node_modules/raw-body/index.js:155:17)
apollo_1   |     at getRawBody (/apollo/node_modules/raw-body/index.js:108:12)
apollo_1   |     at read (/apollo/node_modules/body-parser/lib/read.js:77:3)
apollo_1   |     at jsonParser (/apollo/node_modules/body-parser/lib/types/json.js:135:5)
apollo_1   |     at Layer.handle [as handle_request] (/apollo/node_modules/express/lib/router/layer.js:95:5)
apollo_1   |     at trim_prefix (/apollo/node_modules/express/lib/router/index.js:317:13)
apollo_1   |     at /apollo/node_modules/express/lib/router/index.js:284:7
apollo_1   |     at Function.process_params (/apollo/node_modules/express/lib/router/index.js:335:12)
apollo_1   |     at next (/apollo/node_modules/express/lib/router/index.js:275:10)
apollo_1   |     at cors (/apollo/node_modules/cors/lib/index.js:188:7)
every 2 seconds or so. The server UI also does not seem to be running. Does anyone have an idea what this means?
z

Zanie

10/13/2021, 5:04 PM
Hey Martin, we're investigating this internally. I'll try to get back to you soon.
z

Zach Angell

10/13/2021, 6:08 PM
hi @Martin Felder, are you able to tell from the logs what request is causing this error?
m

Martin Felder

10/14/2021, 8:55 AM
Thank you for having a look at this! Not sure which logs you are talking about? Just the commandline output? I attach that and some other pertinent information.
πŸ‘ 1
Since we had to reinstall our external Postgres server, the database is a clean slate. Same effect happens if I leave out the
--external-postgres
option, so that's not it apparently. There are no flows or agents registered yet.
z

Zach Angell

10/14/2021, 1:20 PM
Thanks for sending the logs!
There are no flows or agents registered yet.
Hmm okay that's good to know. I'm trying to figure out what's hitting the Prefect Server API with large requests πŸ€” I'll try to reproduce on my end
A fresh install of
prefect
0.15.6 +
prefect server start
works as expected for me. Is there something else that could be hitting your server api or any other unique features of your server setup?
m

Martin Felder

10/18/2021, 6:51 AM
Not that I know of - I think I'll try a fresh install, too. The configuration file looks ok to you for a UI visible from the outside? We are always unsure in which sections we have to specify the IP address.
z

Zach Angell

10/18/2021, 1:42 PM
Hmm just based on the logs it looks like something in the UI configuration is off:
:space_invader::space_invader::space_invader: UI running at localhost:8080 :space_invader::space_invader::space_invader:
Maybe the
PayloadTooLargeError
is just a red herring
m

Martin Felder

10/20/2021, 10:04 AM
Ok, you were right - it was a red herring! Apparently there were some LocalAgents running while the DB shut down, and something went awry. After killing the agent and restarting the server, the warnings disappear! Sorry for leading you astray!
But... the UI is still not visible from the outside 😞 With the
config.toml
as stated above and adding the
--expose
flag to
prefect server start
, which I understand I'm supposed to do at 0.15.5+, I'm still getting
Copy code
ui_1       | Missing the PREFECT_SERVER__BASE_URL environment variable.  Using default                                                                                                        
ui_1       | πŸ‘ΎπŸ‘ΎπŸ‘Ύ UI running at localhost:8080 πŸ‘ΎπŸ‘ΎπŸ‘Ύ
Setting
export PREFECT_SERVER__BASE_URL=<http://172.16.18.70>
changes nothing, neither does adding
-v master -uv master
to the server start. Strangely enough, the expose flag does not appear when I run
perfect server start --help
, although I seem to be using the correct executable:
Copy code
$ conda env list
# conda environments:
#
base                     /opt/miniconda
ws_prefect            *  /opt/miniconda/envs/ws_prefect

$ which prefect
/opt/miniconda/envs/ws_prefect/bin/prefect

$ conda list | grep prefect
# packages in environment at /opt/miniconda/envs/ws_prefect:
prefect                   0.15.6             pyhd8ed1ab_0    conda-forge
What am I doing wrong?
z

Zach Angell

10/20/2021, 1:28 PM
Ah looking in the code the
UI running at localhost:8080
message is also a red herring, it doesn't log the actual UI url there
Are you able to see the ui if you run
prefect server start --expose
?
m

Martin Felder

10/20/2021, 2:27 PM
Ha! Indeed I can! Sorry I haven't tried that in my red herring fixation... Thank you very much, problem solved!
z

Zach Angell

10/20/2021, 4:41 PM
Nice! πŸ₯³
2 Views