v

    Viv Ian

    2 years ago
    I’m getting the following error within my app container when attempting to do a remote request. This is part of a kubernetes cluster that also has apollo, hasura, postgres, ui, graphql, and scheduler running. My most recent steps (about 30 minutes ago) were the following: • delete all pods and service in kubernetes •
    docker system prune -a
    for a fresh start 🙂 • created prefect service/deployments for k8s (using latest images) • docker build my app image • created service/deployments for my app for k8s Any ideas? THANKS!
    Zachary Hughes

    Zachary Hughes

    2 years ago
    Hi @Viv Ian, taking a look now!
    It looks like the URL the client is trying to hit might be a bit off-- you're trying to communicate with Prefect Server, correct? Quick clarification: do you have Apollo, Hasura, etc. running all in one service, or are they split into their own kubernetes services?
    v

    Viv Ian

    2 years ago
    Hi @Zachary Hughes, I am trying to communicate with prefect server. The last two lines of my Dockerfile are:
    RUN prefect backend server
    and
    CMD python flow_example.py
    . Apollo, Hasura, etc. are running in their own kubernetes services.
    Zachary Hughes

    Zachary Hughes

    2 years ago
    Okay, gotcha. My immediate thought is that if you've decomposed each part of the stack into its own service, you may need to adjust some of your hosts. Are you able to ping
    <http://graphql:4200/graphql/health>
    from the pod you're trying to register your flow with? I ask that as an arbitrary connectivity test because I'm wondering if the
    localhost
    references aren't playing nicely now that things are in different pods.
    v

    Viv Ian

    2 years ago
    I tested it out by creating a new pod with a container that has the command to
    ping <http://graphql:4200/graphql/health>
    and the response was:
    ping: bad address '<http://graphql:4200/graphql/health>'
    Zachary Hughes

    Zachary Hughes

    2 years ago
    Okay, odd. What about
    <http://1.2.3.4:4200/graphql/health>
    , where 1.2.3.4 is the IP address of your graphql service?
    FWIW, it sounds like you're very nearly there-- just need to sort out this Kubernetes networking issue.
    v

    Viv Ian

    2 years ago
    I’m getting the same error when using the service IP. I also noticed that the port being used for graphql is 4201, so attempted the above steps using port 4201, but still got the same error 😞
    Zachary Hughes

    Zachary Hughes

    2 years ago
    Would you be comfortable posting the YAML for your Apollo and GraphQL service deployments? DMing is also fine if that would be better for you.
    v

    Viv Ian

    2 years ago
    Sure thing! I actually just got them from Josh. 🙂 https://github.com/joshmeek/prefect-server-ui-k8s
    Zachary Hughes

    Zachary Hughes

    2 years ago
    Awesome, thank you! Will take a look.
    Okay, that was actually super helpful in ruling some things out. That YAML all looks pretty solid, so I'm betting this is an issue with the app configuration. When building your app, could you try specifying
    export PREFECT__SERVER__HOST=apollo
    in the same block of instructions where you run
    prefect backend server
    ? It also sounds like building/deploying might be something of a laborious process. If you want to verify that the server API is working as-is before rebuilding your app, you could also
    kubectl port-forward <YOUR APOLLO HERE> 4200:4200
    If the server is working as expected, you should be able to register a flow while port-forwarded to your current setup.