I don’t have any errors in my `apollo` pod logs, b...
# ask-communityw
Wilson Bilkovich
08/27/2021, 1:19 PMI don’t have any errors in my
apollo Copy code
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='prefect-server-initial-apollo.prefect', port=4200): Max retries exceeded with url: /graphql (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused'))n
nicholas
08/27/2021, 3:32 PM
Hi @Wilson Bilkovich - if you ping the apollo pod directly do you get a response?
w
Wilson Bilkovich
08/27/2021, 3:53 PMLet me check; thanks for the reply.
Wilson Bilkovich
08/27/2021, 3:55 PM Copy code
% telnet prefect-server-initial-apollo 4200
Trying 127.1.27.7...
Connected to prefect-server-initial-apollo.
Escape character is '^]'.
GET /graphql
HTTP/1.1 400 Bad Request
X-Powered-By: Express
Access-Control-Allow-Origin: *
Content-Type: text/html; charset=utf-8
Content-Length: 18
ETag: W/"12-7JEJwpG8g89ii7CR/6hhfN27Q+k"
Date: Fri, 27 Aug 2021 15:54:51 GMT
Connection: close
GET query missing.Connection closed by foreign host.n
nicholas
08/27/2021, 4:06 PM
And this agent is the kubernetes agent started with the
--set agent.enabled=truew
Wilson Bilkovich
08/27/2021, 4:39 PMIt is, yeah; using the default tenant also created by the Helm chart
n
nicholas
08/27/2021, 4:52 PM
Are you using any networking policies in your cluster?
nicholas
08/27/2021, 4:52 PM
That might be blocking requests?
w
Wilson Bilkovich
08/27/2021, 5:15 PMI haven’t added anything, let me double-check if there are any. This is Amazon EKS and it does a few things by default.
Wilson Bilkovich
08/27/2021, 5:18 PMNope, no network policies. Also, I can connect from the agent to apollo when I exec into it:
Copy code
root@prefect-server-initial-agent-5f76b4fd69-vn74p:/# telnet prefect-server-initial-apollo 4200
Trying 172.20.65.234...
Connected to prefect-server-initial-apollo.prefect.svc.cluster.local.
Escape character is '^]'.
GET /graphql?query=foo
HTTP/1.1 400 Bad Request
X-Powered-By: Express
Access-Control-Allow-Origin: *
Content-Type: application/json; charset=utf-8
Content-Length: 146
ETag: W/"92-sfDAzDmlWzD691zaMG6MP1fVCE8"
Date: Fri, 27 Aug 2021 17:17:41 GMT
Connection: close
{"errors":[{"message":"Syntax Error: Unexpected Name \"foo\"","locations":[{"line":1,"column":1}],"extensions":{"code":"GRAPHQL_PARSE_FAILED"}}]}
Connection closed by foreign host.n
nicholas
08/27/2021, 6:03 PM
Hm, I'm not entirely sure how to proceed then, if you can query apollo from the rest of the cluster (i'm assuming the UI works) and you can query apollo from the agent container as well I'm at a bit of a loss. Can you start the agent manually ?
w
Wilson Bilkovich
08/27/2021, 6:20 PMHmm, maybe I am just chasing a ghost now, because there’s nothing unhappy in the pod logs for the current instance of the Agent.
n
nicholas
08/27/2021, 6:22 PM
Ah so it's working now? Maybe a network blip?
w
Wilson Bilkovich
08/27/2021, 6:25 PMI’m trying to prove that to myself via Kibana, which I am growing to dislike..
🤔 1
Wilson Bilkovich
08/28/2021, 9:04 PMApparently these logs are from some earlier incarnation of my cluster, because I can’t reproduce it. Thanks for the help, I’ll just assume this was something silly I did in Kubernetes.
✅ 1
n
nicholas
08/30/2021, 7:01 AM
Glad you got it resolved!
50 Views