Hi, I'm running into, what is probably a very simp...
# prefect-community
s
Hi, I'm running into, what is probably a very simple issue with docker networking. Setup: • Orion Server (running locally: 127.0.0.1:4200) • Prefect Agent (running locally) • Airbyte Server (running locally: 127.0.0.1:8000) • Docker Infra Block - Docker running locally The agent can see the server. I can create and apply a deployment. On starting the flow: • The agent kicks off. • The agent creates the docker image. • The agent applies any pip requirements from block env setting. • It then fails with
ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 4200)
I'm assuming the flow running in the docker container cannot see the orion server on port 4200, however, I'm not sure what to do to fix that. No doubt the next failure will be accessing the Airbyte server, but I'm not there yet. Any help is much appreciated.
1
c
Hi Steven, How did you set this up, and did you follow a tutorial of any sort? How is docker being setup and exposed? What happens if you shell into the docker image, and try to curl 127.0.0.1:4200? Do you have logs (flow run logs, agent logs) ? To that point above, can you turn up logging to DEBUG,
prefect config set PREFECT_LOGGING_LEVEL=DEBUG
and share agent logs Also, where is the “ConnectionRefusedError” occurring - is this from the local docker agent?
s
I didn't follow a tutorial. I've read a lot of the documentation, and I'm just building up bit by bit from a fully local stack to a dockerised stack (all local) to eventually move it all to the cloud. It is just part of a learning exercise as part of preparing for some consultancy work.
I'll take a look at your comments later tonight and feed back.
I think my issue is identical to this one: https://github.com/PrefectHQ/prefect/issues/5532 But I've not tried the "bridge" option yet.
c
so you basically want to run everything local, agent, server, and docker ; you want to be able to run a flow from the UI (running locally on 4200) to have a docker container start with the flow (which is also local). Is that accurate?
s
Yes: • Orion Server (running locally: 127.0.0.1:4200) • Prefect Agent (running locally) • Airbyte Server (running locally: 127.0.0.1:8000) • Docker Infra Block actually running the flow (Docker running locally)
It is only to test out the setup and ensure that I fully understand Prefect before I go out to sell it to my clients. Once we can build it all locally, then we can push it piece by piece into the cloud which is the final model we are looking to build, but being able to replicate locally is very valuable not only for understanding, but also for development.
Actually, one step I've made since I posted this is that the code is now in GitHub, so that's not local (I love it by the way - having the code in Github - amazing).
Okay - I've actually solved it by: • Setting the network for the docker container to "bridge" rather than host as per the workaround in this issue (Issue 5532) • Setting the Airbyte server address to "host.docker.internal" in the Airbyte Server block. I'm not sure why I have to set the network to "bridge" though as I had read that "host" would/should work.
🙌 1