Hi All, just upgraded to an Org account and reconfiguring K8's to use the new service accounts - I c...
r
redsquare
10/24/2022, 5:22 PMHi All, just upgraded to an Org account and reconfiguring K8's to use the new service accounts - I cant get my k8s agent to start, the agent looks to be connected fine as it writes out the api url but then 404's on getting the {url}/work_queues
Agent started! Looking for work from queue(s): default-work-queue...Mon, Oct 24 2022 6:17:45 pm17:17:45.536 | ERROR | prefect.agent - Failed to create work queue 'default-work-queue'.Mon, Oct 24 2022 6:17:45 pmprefect.exceptions.PrefectHTTPStatusError: Client error '404 Not Found' for url '<https://api.prefect.cloud/api/accounts/9396b5ff-23c0-42b4-91d9-5892450xxxx/workspaces/de803906-c6f7-45%0A77-a3aa-d300b56fxxx/work_queues/>'👀 1
✅ 1
redsquare
10/24/2022, 5:54 PMIgnore me - on copy paste rancher secrets included a line break in the PREFECT_API_URL
✅ 2
👍 1
g
George Coyne
10/24/2022, 6:01 PMGlad to hear it got sorted out!
🙌 1
r
redsquare
10/24/2022, 7:25 PMPerhaps a better test and console message is needed on connection to see if the url+key is valid rather than 404'ing on the workqueue
c
Christopher Boyd
10/24/2022, 8:55 PMa little late but I noticed in your url, you have %0A which is the url encoded version of a line feed
``````
Christopher Boyd
10/24/2022, 8:56 PM Copy code
de803906-c6f7-45%0A77-a3aa-d300b56fxxx/work_queues/'r
redsquare
10/24/2022, 8:58 PMHey @Christopher Boyd yeah, copy paste issue into rancher from local console 😁
🙌 1
c
Christopher Boyd
10/24/2022, 9:00 PMI think useful to see the outcome and fix even if it’s simple, I doubt this will be the last time we see this happen 🙂
r
redsquare
10/24/2022, 9:03 PMCould a test be done in the agent on connect, currently the logging appears as if it has connected successfully
c
Christopher Boyd
10/24/2022, 9:05 PMLet me check on a few things, Ill get back shortly
Christopher Boyd
10/24/2022, 9:08 PMso If I start an agent fresh, I get the following:
Copy code
Starting v2.6.1 agent connected to
<https://api.prefect.cloud/api/accounts/><redacted>/workspaces/<redacted>...
___ ___ ___ ___ ___ ___ _____ _ ___ ___ _ _ _____
| _ \ _ \ __| __| __/ __|_ _| /_\ / __| __| \| |_ _|
| _/ / _|| _|| _| (__ | | / _ \ (_ | _|| .` | | |
|_| |_|_\___|_| |___\___| |_| /_/ \_\___|___|_|\_| |_|
Agent started! Looking for work from queue(s): kubernetes... Copy code
Starting v2.6.1 agent connected to
<https://api.prefect.cloud/api/accounts/><>/work
spaces/<>...
___ ___ ___ ___ ___ ___ _____ _ ___ ___ _ _ _____
| _ \ _ \ __| __| __/ __|_ _| /_\ / __| __| \| |_ _|
| _/ / _|| _|| _| (__ | | / _ \ (_ | _|| .` | | |
|_| |_|_\___|_| |___\___| |_| /_/ \_\___|___|_|\_| |_|
Agent started! Looking for work from queue(s): kubernetes...
21:07:44.022 | ERROR | prefect.agent - Failed to create work queue 'kubernetes'.
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/prefect/agent.py", line 111, in get_work_queues
work_queue = await self.client.create_work_queue(name=name)
File "/usr/local/lib/python3.9/site-packages/prefect/client/orion.py", line 648, in create_work_queue
response = await <http://self._client.post|self._client.post>("/work_queues/", json=data)
File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1842, in post
return await self.request(
File "/usr/local/lib/python3.9/site-packages/httpx/_client.py", line 1527, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)Christopher Boyd
10/24/2022, 9:11 PMThe client is more or less just reaching the endpoint directly, it doesn’t know if it’s a good or bad URL though since every tenant / workspace will be a different url
Christopher Boyd
10/24/2022, 9:13 PMI can see what the team thinks about this line though, since I can see it being misleading
Copy code
Agent started! Looking for work from queue(s): kubernetes...5 Views
Prefect Community
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
Powered by