Shubham
07/09/2026, 11:36 AMMarvin
07/09/2026, 11:36 AMMarvin
07/09/2026, 11:37 AMPING / PONG lines are normal. They’re just uvicorn/websocket keepalive messages, not the cause by themselves.
What they mean:
- the UI has an open WebSocket to the API
- the server sends a ping
- the browser/client responds with a pong
- this keeps the connection alive
So the main issue is likely server/API responsiveness, not the ping logs.
A few things stand out in your config:
1. You are running with DEBUG logging
- PREFECT_SERVER_LOGGING_LEVEL=DEBUG
- This can create a lot of noise and some overhead
- First step: switch to INFO
2. Your DB env var looks malformed in the pasted config
- You showed:
Environment="PREFECT_API_DATABASE_CONNECTION_URL=<postgresql+asyncpg://prefect:prefectpassword@127.0.0.1:5432/prefect>"
- If those < and > are literally in the service file, that is wrong
- It should be:
Environment="PREFECT_API_DATABASE_CONNECTION_URL=<postgresql+asyncpg://prefect:prefectpassword@127.0.0.1:5432/prefect>"
3. PREFECT_SERVER_API_KEEPALIVE_TIMEOUT=60 is HTTP keepalive-related
- It is not the websocket ping interval you are seeing
- It usually won’t fix intermittent UI slowness
What I’d recommend first:
Update your systemd service like this:
ini
[Unit]
Description=Prefect Server
After=network-online.target postgresql.service
Wants=network-online.target
[Service]
Type=simple
User=root
WorkingDirectory=/home/prefect
Environment="PREFECT_HOME=/home/prefect/.prefect"
Environment="PREFECT_SERVER_API_HOST=192.168.29.151"
Environment="PREFECT_SERVER_API_PORT=4200"
Environment="PREFECT_SERVER_LOGGING_LEVEL=INFO"
Environment="PREFECT_API_DATABASE_CONNECTION_URL=<postgresql+asyncpg://prefect:prefectpassword@127.0.0.1:5432/prefect>"
Environment="PREFECT_SERVER_API_AUTH_STRING=astra:xxxxxxxx"
ExecStart=/home/prefect/.venv/bin/prefect server start --host 0.0.0.0 --port 4200
Restart=on-failure
RestartSec=10
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
Then reload and restart:
bash
sudo systemctl daemon-reload
sudo systemctl restart prefect-server
sudo journalctl -u prefect-server -f
Next checks:
1. Test API health directly
bash
curl -u astra:xxxxxxxx <http://192.168.29.151:4200/api/health>
If that is fast consistently, the backend is mostly OK and the slowdown may be on specific UI queries.
2. Watch for real errors in logs
Look for:
- database timeout errors
- connection pool errors
- postgres errors
- long request times
3. Check PostgreSQL performance
Since you’re self-hosting, intermittent slowness is often DB-related:
- slow queries
- connection pool saturation
- disk I/O / CPU pressure
Useful checks:
bash
top
free -h
df -h
sudo -u postgres psql -c "select now();"
4. Reduce log verbosity
If you don’t need wire-level logs, INFO is better than DEBUG.
A couple more notes:
- The ping/pong messages are expected for the live UI connection.
- If the UI “hangs” while API health is fast, the problem may be a heavy API endpoint or browser/network issue.
- If all API requests are slow, focus on PostgreSQL and server resources first.
Relevant docs:
- Prefect docs
- Self-hosting Prefect
- Settings and profiles
If you want, send:
- output of curl -u ... /api/health
- journalctl -u prefect-server -n 100
- whether the slowness is on all pages or only flow runs / task runs / logs
and I can help narrow it down.Shubham
07/09/2026, 11:56 AMMarvin
07/09/2026, 11:56 AMMarvin
07/09/2026, 11:57 AMrequests, httpx, browser keepalive, or another pooled HTTP client, the first call may hit a dead pooled connection.
Important point:
- the PING/PONG logs you pasted are WebSocket keepalives for the UI
- your failing API calls are likely plain HTTP, not websocket
- so those logs are probably just a symptom of an open UI session, not the cause
What I would check next:
1. Is there a proxy in front of Prefect?
Are clients calling:
- directly: <http://192.168.29.151:4200>
- or through nginx / domain / VPN / LB?
If there is a proxy, this becomes the top suspect.
2. Test from another machine with fresh one-off curl calls
Run this after the server has been idle:
bash
curl -v -u astra:xxxxxxxx <http://192.168.29.151:4200/api/health>
curl -v -u astra:xxxxxxxx <http://192.168.29.151:4200/api/health>
curl -v -u astra:xxxxxxxx <http://192.168.29.151:4200/api/health>
If the first fails and the next ones succeed, please capture the exact error:
- connection refused
- connection reset by peer
- empty reply from server
- timeout
- 401/403
- TLS/SSL error
That error text matters a lot.
3. Force no connection reuse in the test
Try:
bash
curl -v -H "Connection: close" -u astra:xxxxxxxx <http://192.168.29.151:4200/api/health>
If Connection: close makes the issue disappear, that strongly suggests stale keepalive connection reuse.
4. Check if this only happens cross-network
You mentioned “from other network”.
That suggests:
- NAT timeout
- firewall session timeout
- VPN idle timeout
- proxy idle timeout
If local calls from the same host always work, but remote calls after idle fail, the problem is probably between the client and the server, not in Prefect itself.
About your current setting:
ini
Environment="PREFECT_SERVER_API_KEEPALIVE_TIMEOUT=60"
That affects uvicorn HTTP keepalive behavior, but it may not solve the issue if:
- a proxy/firewall kills idle sessions earlier
- the client is reusing dead pooled sockets
- some middlebox has a shorter timeout
Still, 60 is reasonable. The key is that all layers need compatible timeout behavior.
My practical recommendation:
Temporarily simplify the path
1. call Prefect directly by IP and port from the remote client
2. avoid any proxy/domain if possible
3. use curl -v with Connection: close
4. compare behavior
If direct-IP works reliably but proxied/domain access fails after idle, the proxy/network layer is the problem.
Also, switch logs to INFO unless you need packet-level debug:
ini
Environment="PREFECT_SERVER_LOGGING_LEVEL=INFO"
If you want, send me:
- the exact failing curl error output
- whether you use nginx / LB / VPN / firewall in front of Prefect
- whether Connection: close changes the behavior
and I can narrow it down very quickly.Shubham
07/09/2026, 12:02 PM<http://192.168.29.151:4200> and after idl first error our first request gets timedout. we are not using any proxy client is my laptop and server is locally in same networkMarvin
07/09/2026, 12:02 PMMarvin
07/09/2026, 12:02 PM<http://192.168.29.151:4200> directly, with no proxy, and the first request after idle times out, then this is much less likely to be a Prefect-specific setting and more likely one of:
- ARP / neighbor cache resolution delay on the LAN
- host firewall / conntrack / NIC power-saving
- server process briefly not accepting or not responding on first new connection
- less commonly, client connection reuse if you’re using a browser/app session instead of one-off requests
Also: if the browser UI is what times out first after idle, browsers do aggressively reuse connections.
Important: Prefect server itself should not “sleep” after idle.
What to test next
1. Compare browser vs fresh curl
After the server has been idle, from your laptop run:
bash
curl -v --connect-timeout 5 -u astra:xxxxxxxx <http://192.168.29.151:4200/api/health>
Then immediately again:
bash
curl -v --connect-timeout 5 -u astra:xxxxxxxx <http://192.168.29.151:4200/api/health>
And also:
bash
curl -v --connect-timeout 5 -H "Connection: close" -u astra:xxxxxxxx <http://192.168.29.151:4200/api/health>
What matters is whether the timeout happens during:
- TCP connect
- sending request
- waiting for response
If Connection: close helps, that points to stale keepalive reuse.
2. Test from the server itself when idle
On the server:
bash
curl -v --connect-timeout 5 -u astra:xxxxxxxx <http://127.0.0.1:4200/api/health>
If localhost is always fast, but laptop-to-server first request times out, then the problem is definitely network/LAN/firewall/NIC, not Prefect app logic.
3. Check whether the TCP port is listening continuously
On the server:
bash
ss -ltnp | grep 4200
You should see Prefect/uvicorn listening on 0.0.0.0:4200 or similar.
4. Watch packets during the first failed request
On the server, before reproducing:
bash
sudo tcpdump -i any host 192.168.29.151 and port 4200
Then make the first request from your laptop after idle.
This tells you whether:
- the SYN reaches the server
- the server replies
- the request reaches uvicorn
- the response is delayed or absent
5. Check server logs exactly at first timeout
Run:
bash
sudo journalctl -u prefect-server -f
Then reproduce the idle timeout and see whether Prefect logs anything at all for that first request.
Interpretation:
- No log entry at all → request is not reaching the app cleanly; network/socket layer issue
- Log entry appears late → app accepted it but responded slowly
- Immediate auth/error log → app received it, not a network issue
A few strong clues here
Since:
- same LAN
- direct IP
- no proxy
- failure only after idle
- later calls succeed
I would suspect LAN/NIC/firewall behavior before Prefect.
Common examples:
- Linux power-saving on NIC
- firewall state table timeout
- ARP resolution delay
- Wi-Fi sleep behavior on laptop
- security software on either machine
Things worth checking on the server
Firewall
If ufw or iptables/nftables is active:
bash
sudo ufw status
sudo iptables -L -n
sudo nft list ruleset
ARP/neigh table
After idle, before first request and after first request:
bash
ip neigh
If the entry for your laptop/server is stale/delayed, first contact can pause.
NIC power management
If server is on Wi-Fi or certain virtualized NICs, idle wakeup can delay traffic.
Server resource sanity
Even if this doesn’t feel like performance, still check there isn’t a pause:
bash
top
vmstat 1
dmesg -T | tail -n 50
Especially look for:
- NIC resets
- kernel network warnings
- OOM or CPU stall messages
One very useful isolation test
From your laptop, after idle:
bash
ping 192.168.29.151
Then immediately:
bash
curl -v --connect-timeout 5 -u astra:xxxxxxxx <http://192.168.29.151:4200/api/health>
If ping first makes the API request succeed, that strongly suggests ARP/NIC/network-idle behavior, not Prefect.
You can also test:Marvin
07/09/2026, 12:02 PMbash
nc -vz 192.168.29.151 4200
after idle, before curl.
My current best guess
Based on what you’ve shared, my ranking is:
1. network idle / ARP / NIC issue on LAN
2. browser/client reusing stale HTTP connection
3. host firewall / conntrack timeout behavior
4. Prefect/uvicorn issue — possible, but less likely
Docs:
- Self-hosting Prefect
- Settings and profiles
If you want, send me the outputs of these 3 tests:
bash
curl -v --connect-timeout 5 -u astra:xxxxxxxx <http://192.168.29.151:4200/api/health>
curl -v --connect-timeout 5 -H "Connection: close" -u astra:xxxxxxxx <http://192.168.29.151:4200/api/health>
ping 192.168.29.151
right after an idle period, and I can tell you which layer is failing.Shubham
07/09/2026, 12:09 PMMarvin
07/09/2026, 12:09 PMMarvin
07/09/2026, 12:09 PMtcpdump shows is not a new HTTP request. It shows an already-established TCP connection sending tiny packets repeatedly, and the server is ACKing them.
That usually means the client is trying to use an existing idle connection, not opening a fresh one.
Key signs from your capture:
- no TCP SYN packet
- no new handshake
- just packets on an existing connection: 192.168.29.87.62116 > Prefect.4200
- server replies with ACKs
- repeats for a long time
So the likely situation is:
1. your laptop/browser/client has an old keepalive connection open to :4200
2. after idle, the first request tries to reuse that connection
3. something in the HTTP exchange gets stuck or delayed
4. eventually a fresh connection gets created and things work again
This points more toward HTTP keepalive / stale connection reuse than ARP or server sleep.
What I’d do next
1. Capture full packets, not just summary
Run this on the server:
bash
sudo tcpdump -i any -nnvvXSs 0 host 192.168.29.87 and port 4200
Then make exactly one first request after idle.
This will show whether the client is sending:
- partial HTTP headers
- chunked data
- websocket frames
- TCP zero-window / retransmits / FIN/RST
2. Force a brand-new connection from laptop
After idle, do:
bash
curl -v --http1.1 -H "Connection: close" --connect-timeout 5 -u astra:xxxxxxxx <http://192.168.29.151:4200/api/health>
If this succeeds immediately while the browser/UI first call hangs, then the issue is almost certainly stale persistent connection reuse in the browser/client.
3. Compare with browser devtools
Open browser devtools -> Network tab, then after idle reload the UI or call the API.
Look at the first failed request:
- is it marked stalled?
- pending?
- waiting for server response?
- connection start issue?
- does it reuse an existing connection?
That will help confirm whether the browser is trying to reuse a bad socket.
4. Lower or disable keepalive as a test
Since this seems connection-reuse related, test with a shorter keepalive or with connection close behavior at the client side.
You already have:
ini
Environment="PREFECT_SERVER_API_KEEPALIVE_TIMEOUT=60"
For testing, I would actually try lowering it back down or removing it, then restarting, because a longer keepalive can preserve bad idle connections longer in some environments.
Try:
ini
Environment="PREFECT_SERVER_API_KEEPALIVE_TIMEOUT=5"
then:
bash
sudo systemctl daemon-reload
sudo systemctl restart prefect-server
This is not because 5 is universally better, but because it helps test whether long-lived idle sockets are the problem.
5. Test with no browser, only curl
If plain curl with Connection: close works every time after idle, but browser/UI still hangs on first request, then Prefect is probably fine and the issue is specifically browser persistent connection behavior.
What your capture does not suggest
- not an app process crash
- not Prefect “sleeping”
- not a missing listener on port 4200
- not basic connectivity loss
Because the server is actively ACKing traffic on that connection.
Most likely diagnosis now
Based on this packet capture, my best guess is:
the first request after idle is being sent over an old persistent HTTP connection that is in a bad state
Possible reasons:
- browser/client pooled socket reuse
- weird half-open TCP state
- auth/UI session trying to reuse old connection
- less likely a uvicorn/http parser edge case on reused idle sockets
Strongest confirming test
Please run this after idle from the laptop:
bash
curl -v --http1.1 -H "Connection: close" --connect-timeout 5 -u astra:xxxxxxxx <http://192.168.29.151:4200/api/health>
If that works immediately, while the browser still times out on first try, we’ve basically isolated it to persistent connection reuse.
If you want, I can also help you do a cleaner tcpdump interpretation if you send:Marvin
07/09/2026, 12:09 PMcurl -v --http1.1 -H "Connection: close" ...
- browser Network tab screenshot/error text
- result of:
bash
sudo tcpdump -i any -nnvvXSs 0 host 192.168.29.87 and port 4200
One more small note: if you only use this internally, switching the browser test to a private/incognito window can also help eliminate reused connection/session effects.Shubham
07/09/2026, 12:11 PMMarvin
07/09/2026, 12:11 PMMarvin
07/09/2026, 12:12 PMS / S. / . = normal TCP handshake
2. client sends request data
- Flags [P.] ... length 494
3. server ACKs it
4. server sends response data back
- one packet length 174
- another packet length 6271
5. client ACKs the response
6. 4 seconds later the client closes the connection
- client sends F.
- server closes too
That means:
- the request reached Prefect
- Prefect processed it
- Prefect sent a response
- the client received the response packets
- then the client side closed the connection
So for this captured request, the timeout is not on the Prefect server side.
That shifts the likely problem toward your Next.js/frontend side:
- the frontend may be waiting on response parsing
- the frontend may be timing out before consuming the response correctly
- the frontend may be making a request that succeeds at TCP/HTTP level but your app logic treats as failed
- or the browser/Next.js layer is doing multiple requests and only one is succeeding
Most important observation:
there is no network stall in this trace.
The server replies within about `~300 ms`:
- request at 12:10:53.051668
- response starts at 12:10:53.354023
That is healthy.
So now I would focus on what your Next.js code is doing with the response.
Things to check in Next.js
1. Are you calling Prefect from server-side Next.js code or browser code?
- server-side route handler / API route
- client-side fetch in browser
- middleware
- SSR / RSC fetch
This matters because timeouts and connection reuse differ.
2. Log the exact fetch lifecycle
In your Next.js code, log:
- request start time
- when headers arrive
- response status
- response body parse success/failure
- actual thrown error
Something like:
javascript
const start = Date.now();
try {
console.log("Calling Prefect...");
const res = await fetch("<http://192.168.29.151:4200/api/health>", {
headers: {
Authorization: "Basic " + Buffer.from("astra:xxxxxxxx").toString("base64"),
},
cache: "no-store",
});
console.log("Status:", res.status, "after", Date.now() - start, "ms");
const text = await res.text();
console.log("Body received after", Date.now() - start, "ms", text);
} catch (err) {
console.error("Fetch failed after", Date.now() - start, "ms", err);
}
3. Check if you are using an AbortController timeout
A lot of apps add custom timeouts around fetch. If your timeout is too aggressive or not cleaned up correctly, the request can look like a server timeout even when the server responded.
4. Check whether it is actually CORS/auth/body parsing
If browser code is calling Prefect directly, possible issues include:
- CORS preflight behavior
- auth header behavior
- parsing non-JSON body as JSON
- redirect handling
- mixed UI/API requests
5. Compare the exact endpoint
Your capture shows some request succeeded, but maybe the failing one is not /api/health.
The failing endpoint might be:
- /api/...
- /ui/...
- auth-protected endpoint
- websocket upgrade
- large JSON endpoint
If possible, capture the exact failing path.
Best next isolation tests
Test 1: direct curl to same endpoint that Next.js calls
From the laptop:
bash
curl -v -u astra:xxxxxxxx "<http://192.168.29.151:4200/<same-exact-endpoint>>"
If curl succeeds immediately while Next.js times out, the problem is definitely in the app/client layer.
Test 2: use Node fetch outside Next.js
Create a tiny standalone script on the laptop:
```javascript
const start = Date.now();
fetch("http://192.168.29.151:4200/api/health", {
headers: {
Authorization: "Basic " + Buffer.from("astra:xxxxxxxx").toString("base64"),
},
})
.then(async (res) => {
console.log("status", res.status, "ms", Date.now() - start);
console.log(await res.text());
})Marvin
07/09/2026, 12:12 PMIf this works reliably after idle, but Next.js does not, then it is something in Next.js runtime/code path.
*Test 3: inspect the actual HTTP response*
Run:
```bash
sudo tcpdump -i any -nnvvXSs 0 host 192.168.29.87 and port 4200
That will let you see:
- exact request path
- response status code
- response headers
Right now the packet sizes suggest the server probably returned a valid HTTP response, but we haven’t decoded it yet.
My conclusion from your latest capture
Based on this trace alone:
- Prefect accepted the connection
- Prefect received the request
- Prefect sent the response quickly
- the client acknowledged receiving it
- the client later closed the connection
So this specific request does not support “Prefect timed out before responding”.
It suggests the timeout/failure is happening:
- in Next.js fetch handling
- in browser/client logic
- or on a different request than the one captured
If you share the exact Next.js fetch code that calls Prefect, I can probably pinpoint the issue faster. The main things I’d look for are:
- custom timeout wrappers
- await res.json() on non-JSON response
- SSR/client mismatch
- multiple parallel requests
- stale auth header / CORS path confusion