https://prefect.io logo
#prefect-community
Title
# prefect-community
v

Viv Ian

04/21/2020, 5:15 PM
I’m getting the following error within my app container when attempting to do a remote request. This is part of a kubernetes cluster that also has apollo, hasura, postgres, ui, graphql, and scheduler running. My most recent steps (about 30 minutes ago) were the following: • delete all pods and service in kubernetes •
docker system prune -a
for a fresh start 🙂 • created prefect service/deployments for k8s (using latest images) • docker build my app image • created service/deployments for my app for k8s Any ideas? THANKS!
z

Zachary Hughes

04/21/2020, 5:21 PM
Hi @Viv Ian, taking a look now!
It looks like the URL the client is trying to hit might be a bit off-- you're trying to communicate with Prefect Server, correct? Quick clarification: do you have Apollo, Hasura, etc. running all in one service, or are they split into their own kubernetes services?
v

Viv Ian

04/21/2020, 6:16 PM
Hi @Zachary Hughes, I am trying to communicate with prefect server. The last two lines of my Dockerfile are:
RUN prefect backend server
and
CMD python flow_example.py
. Apollo, Hasura, etc. are running in their own kubernetes services.
z

Zachary Hughes

04/21/2020, 6:29 PM
Okay, gotcha. My immediate thought is that if you've decomposed each part of the stack into its own service, you may need to adjust some of your hosts. Are you able to ping
<http://graphql:4200/graphql/health>
from the pod you're trying to register your flow with? I ask that as an arbitrary connectivity test because I'm wondering if the
localhost
references aren't playing nicely now that things are in different pods.
v

Viv Ian

04/21/2020, 7:09 PM
I tested it out by creating a new pod with a container that has the command to
ping <http://graphql:4200/graphql/health>
and the response was:
ping: bad address '<http://graphql:4200/graphql/health>'
z

Zachary Hughes

04/21/2020, 7:13 PM
Okay, odd. What about
<http://1.2.3.4:4200/graphql/health>
, where 1.2.3.4 is the IP address of your graphql service?
FWIW, it sounds like you're very nearly there-- just need to sort out this Kubernetes networking issue.
v

Viv Ian

04/21/2020, 7:40 PM
I’m getting the same error when using the service IP. I also noticed that the port being used for graphql is 4201, so attempted the above steps using port 4201, but still got the same error 😞
z

Zachary Hughes

04/21/2020, 7:48 PM
Would you be comfortable posting the YAML for your Apollo and GraphQL service deployments? DMing is also fine if that would be better for you.
v

Viv Ian

04/21/2020, 7:51 PM
Sure thing! I actually just got them from Josh. 🙂 https://github.com/joshmeek/prefect-server-ui-k8s
z

Zachary Hughes

04/21/2020, 7:52 PM
Awesome, thank you! Will take a look.
Okay, that was actually super helpful in ruling some things out. That YAML all looks pretty solid, so I'm betting this is an issue with the app configuration. When building your app, could you try specifying
export PREFECT__SERVER__HOST=apollo
in the same block of instructions where you run
prefect backend server
? It also sounds like building/deploying might be something of a laborious process. If you want to verify that the server API is working as-is before rebuilding your app, you could also
kubectl port-forward <YOUR APOLLO HERE> 4200:4200
If the server is working as expected, you should be able to register a flow while port-forwarded to your current setup.