Georgiana Ogrean
10/10/2022, 6:38 PMConnectionRefusedError: [Errno 111] Connect call failed ('0.0.0.0', 4200)
The Orion UI comes up just fine when I do not try to build a deployment with:
prefect deployment build /app/my_package/flows/my_python_script.py:my_flow_name --name generate_deployment --tag dev --infra docker-container
Any ideas what I could be missing? More details in the thread.prefect config set PREFECT_API_URL=<http://0.0.0.0:4200/api>
prefect config set PREFECT_ORION_UI_API_URL=http://"${host_ip}":4200/api
Then the Orion UI is started with:
prefect orion start --host=0.0.0.0
And this all works fine as long as I don’t build a deployment.Traceback (most recent call last):
File "/app/venv/lib/python3.9/site-packages/anyio/_core/_sockets.py", line 164, in try_connect
stream = await asynclib.connect_tcp(remote_host, remote_port, local_address)
File "/app/venv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 1691, in connect_tcp
await get_running_loop().create_connection(
File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1056, in create_connection
raise exceptions[0]
File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1041, in create_connection
sock = await self._connect_sock(
File "/usr/local/lib/python3.9/asyncio/base_events.py", line 955, in _connect_sock
await self.sock_connect(sock, address)
File "/usr/local/lib/python3.9/asyncio/selector_events.py", line 502, in sock_connect
return await fut
File "/usr/local/lib/python3.9/asyncio/selector_events.py", line 537, in _sock_connect_cb
raise OSError(err, f'Connect call failed {address}')
ConnectionRefusedError: [Errno 111] Connect call failed ('0.0.0.0', 4200)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/venv/lib/python3.9/site-packages/httpcore/_exceptions.py", line 8, in map_exceptions
yield
File "/app/venv/lib/python3.9/site-packages/httpcore/backends/asyncio.py", line 109, in connect_tcp
stream: anyio.abc.ByteStream = await anyio.connect_tcp(
File "/app/venv/lib/python3.9/site-packages/anyio/_core/_sockets.py", line 222, in connect_tcp
raise OSError("All connection attempts failed") from cause
OSError: All connection attempts failed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/venv/lib/python3.9/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
yield
File "/app/venv/lib/python3.9/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
resp = await self._pool.handle_async_request(req)
File "/app/venv/lib/python3.9/site-packages/httpcore/_async/connection_pool.py", line 253, in handle_async_request
raise exc
File "/app/venv/lib/python3.9/site-packages/httpcore/_async/connection_pool.py", line 237, in handle_async_request
response = await connection.handle_async_request(request)
File "/app/venv/lib/python3.9/site-packages/httpcore/_async/connection.py", line 86, in handle_async_request
raise exc
File "/app/venv/lib/python3.9/site-packages/httpcore/_async/connection.py", line 63, in handle_async_request
stream = await self._connect(request)
File "/app/venv/lib/python3.9/site-packages/httpcore/_async/connection.py", line 111, in _connect
stream = await self._network_backend.connect_tcp(**kwargs)
File "/app/venv/lib/python3.9/site-packages/httpcore/backends/auto.py", line 29, in connect_tcp
return await self._backend.connect_tcp(
File "/app/venv/lib/python3.9/site-packages/httpcore/backends/asyncio.py", line 109, in connect_tcp
stream: anyio.abc.ByteStream = await anyio.connect_tcp(
File "/usr/local/lib/python3.9/contextlib.py", line 137, in __exit__
self.gen.throw(typ, value, traceback)
File "/app/venv/lib/python3.9/site-packages/httpcore/_exceptions.py", line 12, in map_exceptions
raise to_exc(exc)
httpcore.ConnectError: All connection attempts failed
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/venv/lib/python3.9/site-packages/prefect/cli/_utilities.py", line 41, in wrapper
return fn(*args, **kwargs)
File "/app/venv/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 201, in coroutine_wrapper
return run_async_in_new_loop(async_fn, *args, **kwargs)
File "/app/venv/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 152, in run_async_in_new_loop
return anyio.run(partial(__fn, *args, **kwargs))
File "/app/venv/lib/python3.9/site-packages/anyio/_core/_eventloop.py", line 70, in run
return asynclib.run(func, *args, **backend_options)
File "/app/venv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 292, in run
return native_run(wrapper(), debug=debug)
File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/local/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/app/venv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
return await func(*args)
File "/app/venv/lib/python3.9/site-packages/prefect/cli/deployment.py", line 719, in build
deployment = await Deployment.build_from_flow(
File "/app/venv/lib/python3.9/site-packages/prefect/deployments.py", line 679, in build_from_flow
await deployment.load()
File "/app/venv/lib/python3.9/site-packages/prefect/deployments.py", line 483, in load
deployment = await client.read_deployment_by_name(
File "/app/venv/lib/python3.9/site-packages/prefect/client/orion.py", line 1280, in read_deployment_by_name
response = await self._client.get(f"/deployments/name/{name}")
File "/app/venv/lib/python3.9/site-packages/httpx/_client.py", line 1751, in get
return await self.request(
File "/app/venv/lib/python3.9/site-packages/httpx/_client.py", line 1527, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
File "/app/venv/lib/python3.9/site-packages/prefect/client/base.py", line 159, in send
await super().send(*args, **kwargs)
File "/app/venv/lib/python3.9/site-packages/httpx/_client.py", line 1614, in send
response = await self._send_handling_auth(
File "/app/venv/lib/python3.9/site-packages/httpx/_client.py", line 1642, in _send_handling_auth
response = await self._send_handling_redirects(
File "/app/venv/lib/python3.9/site-packages/httpx/_client.py", line 1679, in _send_handling_redirects
response = await self._send_single_request(request)
File "/app/venv/lib/python3.9/site-packages/httpx/_client.py", line 1716, in _send_single_request
response = await transport.handle_async_request(request)
File "/app/venv/lib/python3.9/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
resp = await self._pool.handle_async_request(req)
File "/usr/local/lib/python3.9/contextlib.py", line 137, in __exit__
self.gen.throw(typ, value, traceback)
File "/app/venv/lib/python3.9/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectError: All connection attempts failed
An exception occurred.
Zanie
10/10/2022, 7:22 PMPREFECT_ORION_UI_API_URL
.PREFECT_ORION_UI_API_URL
from the PREFECT_API_URL
if it’s not set.Georgiana Ogrean
10/10/2022, 8:20 PMPREFECT_API_URL
and now seeing a different exception when trying to create the deployment:
Traceback (most recent call last):
File "/app/venv/lib/python3.9/site-packages/httpcore/backends/asyncio.py", line 109, in connect_tcp
stream: anyio.abc.ByteStream = await anyio.connect_tcp(
File "/app/venv/lib/python3.9/site-packages/anyio/_core/_sockets.py", line 218, in connect_tcp
await event.wait()
File "/app/venv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 662, in __aexit__
raise exceptions[0]
File "/app/venv/lib/python3.9/site-packages/anyio/_core/_sockets.py", line 164, in try_connect
stream = await asynclib.connect_tcp(remote_host, remote_port, local_address)
File "/app/venv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 1691, in connect_tcp
await get_running_loop().create_connection(
File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1041, in create_connection
sock = await self._connect_sock(
File "/usr/local/lib/python3.9/asyncio/base_events.py", line 955, in _connect_sock
await self.sock_connect(sock, address)
File "/usr/local/lib/python3.9/asyncio/selector_events.py", line 502, in sock_connect
return await fut
asyncio.exceptions.CancelledError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/venv/lib/python3.9/site-packages/httpcore/_exceptions.py", line 8, in map_exceptions
yield
File "/app/venv/lib/python3.9/site-packages/httpcore/backends/asyncio.py", line 109, in connect_tcp
stream: anyio.abc.ByteStream = await anyio.connect_tcp(
File "/app/venv/lib/python3.9/site-packages/anyio/_core/_tasks.py", line 118, in __exit__
raise TimeoutError
TimeoutError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/venv/lib/python3.9/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
yield
File "/app/venv/lib/python3.9/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
resp = await self._pool.handle_async_request(req)
File "/app/venv/lib/python3.9/site-packages/httpcore/_async/connection_pool.py", line 253, in handle_async_request
raise exc
File "/app/venv/lib/python3.9/site-packages/httpcore/_async/connection_pool.py", line 237, in handle_async_request
response = await connection.handle_async_request(request)
File "/app/venv/lib/python3.9/site-packages/httpcore/_async/connection.py", line 86, in handle_async_request
raise exc
File "/app/venv/lib/python3.9/site-packages/httpcore/_async/connection.py", line 63, in handle_async_request
stream = await self._connect(request)
File "/app/venv/lib/python3.9/site-packages/httpcore/_async/connection.py", line 111, in _connect
stream = await self._network_backend.connect_tcp(**kwargs)
File "/app/venv/lib/python3.9/site-packages/httpcore/backends/auto.py", line 29, in connect_tcp
return await self._backend.connect_tcp(
File "/app/venv/lib/python3.9/site-packages/httpcore/backends/asyncio.py", line 109, in connect_tcp
stream: anyio.abc.ByteStream = await anyio.connect_tcp(
File "/usr/local/lib/python3.9/contextlib.py", line 137, in __exit__
self.gen.throw(typ, value, traceback)
File "/app/venv/lib/python3.9/site-packages/httpcore/_exceptions.py", line 12, in map_exceptions
raise to_exc(exc)
httpcore.ConnectTimeout
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/venv/lib/python3.9/site-packages/prefect/cli/_utilities.py", line 41, in wrapper
return fn(*args, **kwargs)
File "/app/venv/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 201, in coroutine_wrapper
return run_async_in_new_loop(async_fn, *args, **kwargs)
File "/app/venv/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 152, in run_async_in_new_loop
return anyio.run(partial(__fn, *args, **kwargs))
File "/app/venv/lib/python3.9/site-packages/anyio/_core/_eventloop.py", line 70, in run
return asynclib.run(func, *args, **backend_options)
File "/app/venv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 292, in run
return native_run(wrapper(), debug=debug)
File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/local/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/app/venv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
return await func(*args)
File "/app/venv/lib/python3.9/site-packages/prefect/cli/deployment.py", line 719, in build
deployment = await Deployment.build_from_flow(
File "/app/venv/lib/python3.9/site-packages/prefect/deployments.py", line 679, in build_from_flow
await deployment.load()
File "/app/venv/lib/python3.9/site-packages/prefect/deployments.py", line 483, in load
deployment = await client.read_deployment_by_name(
File "/app/venv/lib/python3.9/site-packages/prefect/client/orion.py", line 1280, in read_deployment_by_name
response = await self._client.get(f"/deployments/name/{name}")
File "/app/venv/lib/python3.9/site-packages/httpx/_client.py", line 1751, in get
return await self.request(
File "/app/venv/lib/python3.9/site-packages/httpx/_client.py", line 1527, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
File "/app/venv/lib/python3.9/site-packages/prefect/client/base.py", line 159, in send
await super().send(*args, **kwargs)
File "/app/venv/lib/python3.9/site-packages/httpx/_client.py", line 1614, in send
response = await self._send_handling_auth(
File "/app/venv/lib/python3.9/site-packages/httpx/_client.py", line 1642, in _send_handling_auth
response = await self._send_handling_redirects(
File "/app/venv/lib/python3.9/site-packages/httpx/_client.py", line 1679, in _send_handling_redirects
response = await self._send_single_request(request)
File "/app/venv/lib/python3.9/site-packages/httpx/_client.py", line 1716, in _send_single_request
response = await transport.handle_async_request(request)
File "/app/venv/lib/python3.9/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
resp = await self._pool.handle_async_request(req)
File "/usr/local/lib/python3.9/contextlib.py", line 137, in __exit__
self.gen.throw(typ, value, traceback)
File "/app/venv/lib/python3.9/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectTimeout
An exception occurred.
Jeff Hale
10/12/2022, 3:30 AMGeorgiana Ogrean
10/12/2022, 11:45 PMwsgi.py
file in the project directory:
if __name__ == "__main__":
app.run(host='0.0.0.0', port=9695, use_reloader=False)
The app is created in the __init__
file of the project directory with:
def init_app():
"""Create a Flask app with an embedded Dash app."""
app = Flask(__name__, instance_relative_config=False)
app.config.from_object('config.Config')
with app.app_context():
from . import routes
# Import Dash application
from .dashboard import init_dashboard
app = init_dashboard(app)
return app
so it also brings up a Dash dashboard, if that makes any difference.
In the top directory, there is a Dockerfile
that creates a virtual environment, installs some dependencies, changes the PATH
environment variable inside the container, and then executes a bunch of commands in a docker-entrypoint.py
file. So something like:
RUN . venv/bin/activate \
&& poetry install --only main
ENV PATH="/app/venv/bin:$PATH"
ENTRYPOINT ["/app/docker-entrypoint.sh"]
The Orion UI is started from the docker-entrypoint.sh
file, and that is also where the app is started by executing the code in `wsgi.py`:
#! /bin/bash
# TODO: this is embarrassing, but will sort it out later
host_ip=$(curl <http://checkip.amazonaws.com>)
prefect config set PREFECT_API_URL=<http://0.0.0.0:4200/api>
prefect config set PREFECT_ORION_UI_API_URL=http://"${host_ip}":4200/api
prefect profile use default
# run the Orion UI in the background
prefect orion start --host=0.0.0.0 &
# TODO: this fails!
prefect deployment build /app/meerkat_statistics/flows/monthly_reports.py:generate_report --name generate_report_deployment --tag dev
python3 wsgi.py
I tried changing PREFECT_API_URL
to http://"${host_ip}":4200/api
, but by doing that the workflows triggered/run were no longer showing up in the Orion UI. So I switched that back to 0.0.0.0:4200/api
. With either value of PREFECT_API_URL
though, prefect deployment build ...
fails with one of the two errors above.