https://prefect.io logo
Title
k

kiran

05/19/2023, 5:26 PM
Hi y'all. I have a script running that seems to be functioning alright — the logs in Cloud don't any issues. But my piped output shows
16:50:25.956 | ERROR   | prefect._internal.concurrency.services - Service 'EventsWorker' failed.
Any idea what's going on?
1
z

Zanie

05/19/2023, 5:26 PM
Can you share the traceback please?
k

kiran

05/19/2023, 5:27 PM
It's happening multiple times, but the initial subflows have completed successfully. This is from the latest subflow that Cloud shows as still running
16:50:26.347 | INFO    | Flow run 'armored-swan' - Executing 1 SQL statements
16:50:26.422 | ERROR   | prefect._internal.concurrency.services - Service 'EventsWorker' failed.
Traceback (most recent call last):
  File "/home/.../.venv/lib/python3.10/site-packages/prefect/_internal/concurrency/services.py", line 113, in _run
    async with self._lifespan():
  File "/home/dair/.pyenv/versions/3.10.0/lib/python3.10/contextlib.py", line 199, in __aenter__
    return await anext(self.gen)
  File "/home/.../.venv/lib/python3.10/site-packages/prefect/events/worker.py", line 30, in _lifespan
    async with self._client:
  File "/home/.../.venv/lib/python3.10/site-packages/prefect/events/clients.py", line 118, in __aenter__
    await self._reconnect()
  File "/home/.../.venv/lib/python3.10/site-packages/prefect/events/clients.py", line 136, in _reconnect
    self._websocket = await self._connect.__aenter__()
  File "/home/.../.venv/lib/python3.10/site-packages/websockets/legacy/client.py", line 637, in __aenter__
    return await self
  File "/home/.../.venv/lib/python3.10/site-packages/websockets/legacy/client.py", line 655, in __await_impl_timeout__
    return await self.__await_impl__()
  File "/home/.../.venv/lib/python3.10/site-packages/websockets/legacy/client.py", line 662, in __await_impl__
    await protocol.handshake(
  File "/home/.../.venv/lib/python3.10/site-packages/websockets/legacy/client.py", line 329, in handshake
    raise InvalidStatusCode(status_code, response_headers)
websockets.exceptions.InvalidStatusCode: server rejected WebSocket connection: HTTP 403
z

Zanie

05/19/2023, 6:28 PM
This is an error in a background service so it wont fail your runs
cc @Chris Pickett regarding the websocket connection failure
:thank_you: 1
k

kiran

05/19/2023, 6:28 PM
OK thanks. So, I shouldn't worry about it?
z

Zanie

05/19/2023, 6:29 PM
You’ll be missing events since the events service is failing
It’s not a big deal for your runs, but we’d love to fix it 🙂
c

Chris Pickett

05/19/2023, 6:29 PM
cc: @Chris Guidry
Can you share your account and workspace id so we can look into this?
👍🏾 1