Hi Guys, I am not able to create work pool du...
# ask-communityb
Brijesh
06/06/2023, 4:47 PMHi Guys, I am not able to create work pool due to below ssl error in prefect server. The error only occurs in workpool creation probably happens when prefect server is connecting to raw.githubusercontent.com 163917.540 | ERROR | prefect.server - Encountered exception in request:
✅ 1
b
Bianca Hoch
06/06/2023, 5:17 PMHey! Thank you for raising this. Would you mind moving the stack trace into the thread so that we can keep the channel tidy? Thanks in advance!
Bianca Hoch
06/06/2023, 5:18 PMAlso could you tell us a bit more about the environment that you're running Prefect in? Are you using a corporate VPN, or firewall of somesort?
Bianca Hoch
06/06/2023, 5:21 PMJust a heads up that I'm not 100% familiar with resolving cert issues, but I did find some resources for you to take a preliminary look at:
• Common SSL Issues on Python and How to Fix it
• Worker types not available if GitHub is unreachable
• Setting an SSL_CERT_FILE
b
Brijesh
06/06/2023, 5:28 PMHi Bianca, I did debugging, and found github redirect to be the cause, httpx client fails when there is redirect, passing follow redirect = True in httpx.asncclient fixes this issue, I am using 2.10.12 with python 3.11 on ubuntu. Here is more detail on error => raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Redirect response '302 Moved Temporarily' for url 'https://raw.githubusercontent.com/PrefectHQ/prefect-collection-registry/main/views/aggregate-worker-metadata.json'
Redirect location: 'https://raw.githubusercontent.com.x.f621e9480a478040990b270098c47c323ced.9270fc4a.id.opendns.com/s/raw.githubusercontent.com/PrefectHQ/prefect-collection-registry/main/views/aggregate-worker-metadata.json?X-OpenDNS-Session=_f621e9480a478040990b270098c47c323ced9270fc4a_Hsu8LRw2_'
✅ 1
b
Bianca Hoch
06/06/2023, 6:07 PMWonderful, thanks for sharing the solution here as well! Glad you were able to determine the cause.
Bianca Hoch
06/06/2023, 6:11 PMWhen you get a chance, would you mind editing the body of your message and removing the error trace? I'll post the error that you shared here so that other people can reference it in the future:
Copy code
16:39:17.540 | ERROR | prefect.server - Encountered exception in request:
Traceback (most recent call last):
File "C:\apps\python_3_11\Lib\site-packages\cachetools\__init__.py", line 68, in _getitem
return self.__data[key]
~~~~~~~~~~~^^^^^
KeyError: 'aggregate-worker-metadata'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\apps\python_3_11\Lib\site-packages\prefect\server\api\collections.py", line 31, in get_collection_view
return GLOBAL_COLLECTIONS_VIEW_CACHE[view]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\cachetools\__init__.py", line 418, in _getitem
return cache_getitem(self, key)
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\cachetools\__init__.py", line 70, in _getitem
return self.__missing__(key)
^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\cachetools\__init__.py", line 97, in _missing
raise KeyError(key)
KeyError: 'aggregate-worker-metadata'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\apps\python_3_11\Lib\site-packages\starlette\middleware\errors.py", line 162, in _call
await <http://self.app|self.app>(scope, receive, _send)
File "C:\apps\python_3_11\Lib\site-packages\starlette\middleware\gzip.py", line 24, in _call
await responder(scope, receive, send)
File "C:\apps\python_3_11\Lib\site-packages\starlette\middleware\gzip.py", line 44, in _call
await <http://self.app|self.app>(scope, receive, self.send_with_gzip)
File "C:\apps\python_3_11\Lib\site-packages\starlette\middleware\exceptions.py", line 79, in _call
raise exc
File "C:\apps\python_3_11\Lib\site-packages\starlette\middleware\exceptions.py", line 68, in _call
await <http://self.app|self.app>(scope, receive, sender)
File "C:\apps\python_3_11\Lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in _call
raise e
File "C:\apps\python_3_11\Lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in _call
await <http://self.app|self.app>(scope, receive, send)
File "C:\apps\python_3_11\Lib\site-packages\starlette\routing.py", line 718, in _call
await route.handle(scope, receive, send)
File "C:\apps\python_3_11\Lib\site-packages\starlette\routing.py", line 276, in handle
await <http://self.app|self.app>(scope, receive, send)
File "C:\apps\python_3_11\Lib\site-packages\starlette\routing.py", line 66, in app
response = await func(request)
^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\prefect\server\utilities\server.py", line 103, in handle_response_scoped_depends
response = await default_handler(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\fastapi\routing.py", line 237, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\fastapi\routing.py", line 163, in run_endpoint_function
return await dependant.call(**values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\prefect\server\api\collections.py", line 18, in read_view_content
return await get_collection_view(view)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\prefect\server\api\collections.py", line 43, in get_collection_view
resp = await client.get(repo_url + f"/views/{view_filename}")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\httpx\_client.py", line 1757, in get
return await self.request(
^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\httpx\_client.py", line 1530, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\httpx\_client.py", line 1617, in send
response = await self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\httpx\_client.py", line 1645, in _send_handling_auth
response = await self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\httpx\_client.py", line 1682, in _send_handling_redirects
response = await self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\httpx\_client.py", line 1719, in _send_single_request
response = await transport.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\httpx\_transports\default.py", line 353, in handle_async_request
resp = await self._pool.handle_async_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\httpcore\_async\connection_pool.py", line 261, in handle_async_request
raise exc
File "C:\apps\python_3_11\Lib\site-packages\httpcore\_async\connection_pool.py", line 245, in handle_async_request
response = await connection.handle_async_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\httpcore\_async\connection.py", line 92, in handle_async_request
raise exc
File "C:\apps\python_3_11\Lib\site-packages\httpcore\_async\connection.py", line 69, in handle_async_request
stream = await self._connect(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\httpcore\_async\connection.py", line 149, in _connect
stream = await stream.start_tls(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\httpcore\backends\asyncio.py", line 78, in start_tls
raise exc
File "C:\apps\python_3_11\Lib\site-packages\httpcore\backends\asyncio.py", line 69, in start_tls
ssl_stream = await anyio.streams.tls.TLSStream.wrap(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\anyio\streams\tls.py", line 123, in wrap
await wrapper._call_sslobject_method(ssl_object.do_handshake)
File "C:\apps\python_3_11\Lib\site-packages\anyio\streams\tls.py", line 131, in _call_sslobject_method
result = func(*args)
^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\ssl.py", line 979, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)
Exception in ASGI application
Traceback (most recent call last):
File "C:\apps\python_3_11\Lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 428, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\uvicorn\middleware\proxy_headers.py", line 78, in _call
return await <http://self.app|self.app>(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\apps\python_3_11\Lib\site-packages\uvicorn\middleware\message_logger.py", line 86, in _call
raise exc from None
File "C:\apps\python_3_11\Lib\site-packages\uvicorn\middleware\message_logger.py", line 82, in _call
await <http://self.app|self.app>(scope, inner_receive, inner_send)
File "C:\apps\python_3_11\Lib\site-packages\fastapi\applications.py", line 276, in _call
await super().__call__(scope, receive, send)
File "C:\apps\python_3_11\Lib\site-packages\starlette\applications.py", line 122, in _call
await self.middleware_stack(scope, receive, send)
File "C:\apps\python_3_11\Lib\site-packages\starlette\middleware\errors.py", line 184, in _call
raise exc
File "C:\apps\python_3_11\Lib\site-packages\starlette\middleware\errors.py", line 162, in _call
await <http://self.app|self.app>(scope, receive, _send)
File "C:\apps\python_3_11\Lib\site-packages\prefect\server\api\server.py", line 121, in _call
await <http://self.app|self.app>(scope, receive, send)
File "C:\apps\python_3_11\Lib\site-packages\starlette\middleware\cors.py", line 83, in _call
await <http://self.app|self.app>(scope, receive, send)
File "C:\apps\python_3_11\Lib\site-packages\starlette\middleware\exceptions.py", line 79, in _call
raise exc
File "C:\apps\python_3_11\Lib\site-packages\starlette\middleware\exceptions.py", line 68, in _call
await <http://self.app|self.app>(scope, receive, sender)
File "C:\apps\python_3_11\Lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in _call
raise e
File "C:\apps\python_3_11\Lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in _call
await <http://self.app|self.app>(scope, receive, send)
File "C:\apps\python_3_11\Lib\site-packages\starlette\routing.py", line 718, in _call
await route.handle(scope, receive, send)
File "C:\apps\python_3_11\Lib\site-packages\starlette\routing.py", line 443, in handle
await <http://self.app|self.app>(scope, receive, send)
File "C:\apps\python_3_11\Lib\site-packages\fastapi\applications.py", line 276, in _call
await super().__call__(scope, receive, send)
File "C:\apps\python_3_11\Lib\site-packages\starlette\applications.py", line 122, in _call
await self.middleware_stack(scope, receive, send)
File "C:\apps\python_3_11\Lib\site-packages\starlette\middleware\errors.py", line 184, in _call
_____________________________n
Nikhil Jain
09/19/2023, 11:02 PMNone of the suggestions in the links posted above worked for me. In particular the first link (Common SSL Issues on Python and How to Fix it) is junk… all the commands mentioned in that article are wrong.
On MacOS, for me the fix that worked was this:
Copy code
pip install certifi
/Applications/Python\ 3.9/Install\ Certificates.command14 Views