Hi I m having some issues with accessing prefect 2 0 cloud f Prefect Community #prefect-community

Hi, I'm having some issues with accessing prefect ...

Tim-Oliver

10/18/2022, 1:21 PM

Hi, I'm having some issues with accessing prefect 2.0 cloud from behind a proxy. I can login and switch workspaces but

prefect block ls

times out. If I switch to a network without proxy everything works. Any suggestions?

✅ 1

Tim-Oliver

10/18/2022, 1:22 PM

prefect cloud workspace ls

works.

Tim-Oliver

10/18/2022, 1:45 PM

Copy code

(prefect2)$ prefect block ls
Traceback (most recent call last):
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpcore/backends/asyncio.py", line 109, in connect_tcp
    stream: anyio.abc.ByteStream = await anyio.connect_tcp(
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/anyio/_core/_sockets.py", line 218, in connect_tcp
    await event.wait()
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 662, in __aexit__
    raise exceptions[0]
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/anyio/_core/_sockets.py", line 164, in try_connect
    stream = await asynclib.connect_tcp(remote_host, remote_port, local_address)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 1691, in connect_tcp
    await get_running_loop().create_connection(
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/asyncio/base_events.py", line 1050, in create_connection
    sock = await self._connect_sock(
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/asyncio/base_events.py", line 961, in _connect_sock
    await self.sock_connect(sock, address)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/asyncio/selector_events.py", line 500, in sock_connect
    return await fut
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpcore/_exceptions.py", line 8, in map_exceptions
    yield
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpcore/backends/asyncio.py", line 109, in connect_tcp
    stream: anyio.abc.ByteStream = await anyio.connect_tcp(
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/anyio/_core/_tasks.py", line 118, in __exit__
    raise TimeoutError
TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
    yield
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpcore/_async/connection_pool.py", line 253, in handle_async_request
    raise exc
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpcore/_async/connection_pool.py", line 237, in handle_async_request
    response = await connection.handle_async_request(request)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpcore/_async/connection.py", line 86, in handle_async_request
    raise exc
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpcore/_async/connection.py", line 63, in handle_async_request
    stream = await self._connect(request)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpcore/_async/connection.py", line 111, in _connect
    stream = await self._network_backend.connect_tcp(**kwargs)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpcore/backends/auto.py", line 29, in connect_tcp
    return await self._backend.connect_tcp(
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpcore/backends/asyncio.py", line 109, in connect_tcp
    stream: anyio.abc.ByteStream = await anyio.connect_tcp(
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpcore/_exceptions.py", line 12, in map_exceptions
    raise to_exc(exc)
httpcore.ConnectTimeout

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/prefect/cli/_utilities.py", line 41, in wrapper
    return fn(*args, **kwargs)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 201, in coroutine_wrapper
    return run_async_in_new_loop(async_fn, *args, **kwargs)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 152, in run_async_in_new_loop
    return anyio.run(partial(__fn, *args, **kwargs))
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/anyio/_core/_eventloop.py", line 70, in run
    return asynclib.run(func, *args, **backend_options)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 292, in run
    return native_run(wrapper(), debug=debug)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
    return await func(*args)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/prefect/cli/block.py", line 176, in block_ls
    blocks = await client.read_block_documents()
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/prefect/client/orion.py", line 1160, in read_block_documents
    response = await <http://self._client.post|self._client.post>(
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpx/_client.py", line 1842, in post
    return await self.request(
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpx/_client.py", line 1527, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/prefect/client/base.py", line 159, in send
    await super().send(*args, **kwargs)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpx/_client.py", line 1614, in send
    response = await self._send_handling_auth(
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpx/_client.py", line 1642, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpx/_client.py", line 1679, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpx/_client.py", line 1716, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/tibuch/Apps/miniconda3/envs/prefect2/lib/python3.9/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectTimeout
An exception occurred.

Christopher Boyd

10/18/2022, 1:48 PM

Hi Tim - are you able to access it from a separate network just fine? If you’re behind a proxy, then I’d suggest whitelisting / allowing the prefect domain to whoever manages the proxy. I don’t quite think this is a prefect issue though

Tim-Oliver

10/18/2022, 1:52 PM

From a separate network it works. I can ask them, but they are very strict with their rules here... Interestingly I never had such an issue with Prefect 1.

Tim-Oliver

10/18/2022, 1:58 PM

Could you suggest some address that I could test-ping the connection?

Christopher Boyd

10/18/2022, 2:07 PM

The API?

<http://api.prefect.io|api.prefect.io>

Christopher Boyd

10/18/2022, 2:08 PM

it should be your prefect api url minus the tenant / workspace

Christopher Boyd

10/18/2022, 2:08 PM

and prefect 1 was a separate URL

Christopher Boyd

10/18/2022, 2:15 PM

you should be able to curl -vvv https://api.prefect.io - with no parameter, you should get a response:

Copy code

Connection #0 to host <http://api.prefect.io|api.prefect.io> left intact
GET query missing.%

If you are behind a proxy, it should show in the session connection and a cache miss / response from the proxy

Christopher Boyd

10/18/2022, 2:16 PM

you can also check your proxy settings:

echo $HTTP_PROXY && echo $HTTPS_PROXY

Christopher Boyd

10/18/2022, 2:18 PM

you can probably set a no_proxy locally as well: export

no_proxy=*.<http://prefect.io|prefect.io>,<http://api.prefect.io|api.prefect.io>

Tim-Oliver

10/18/2022, 2:18 PM

Copy code

(prefect2)❯ curl -vvv <https://api.prefect.io>
* Uses proxy env variable https_proxy == '<http://our-proxy.ch:8080/>'
*   Trying 172.27.28.7:8080...
* Connected to <http://our-proxy.ch|our-proxy.ch> (172.27.28.7) port 8080 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to <http://api.prefect.io:443|api.prefect.io:443>
> CONNECT <http://api.prefect.io:443|api.prefect.io:443> HTTP/1.1
> Host: <http://api.prefect.io:443|api.prefect.io:443>
> User-Agent: curl/7.81.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< 
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1

Tim-Oliver

10/18/2022, 2:21 PM

If I set

no_proxy

I get:

Copy code

(prefect2)❯ curl -vvv <https://api.prefect.io>           
* Uses proxy env variable no_proxy == '*.<http://prefect.io|prefect.io>,<http://api.prefect.io|api.prefect.io>'
*   Trying 34.111.157.195:443...
*   Trying 2600:1901:0:cb3c:::443...
* Immediate connect fail for 2600:1901:0:cb3c::: Network is unreachable

Which makes sense.

Zanie

10/18/2022, 2:27 PM

There's an issue to add a setting for this to only use the proxy for traffic from the Prefect client

Tim-Oliver

10/18/2022, 2:37 PM

This one: https://github.com/PrefectHQ/prefect/issues/7204 ?

8 Views

Open in Slack

Previous Next