https://prefect.io
Join Slack
Hello everyone, I am not sure if this is correct place to ask this question, but today suddenly all ...
# ask-community
h
Hello everyone, I am not sure if this is correct place to ask this question, but today suddenly all of our prefect flows started failing and raise following error 
(401) Reason: Unauthorized HTTP response headers: HTTPHeaderDict({'Audit-Id': '', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Date': 'Tue, 20 Dec 2022 13:00:03 GMT', 'Content-Length': '129'}) HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Unauthorized","reason":"Unauthorized","code":401}
I have checked all keys and non of them is expired What could be other possible reason for this error ?
👀 1
✅ 1
@Bob LeBoeuf
b
@Hamza Kazmi I just sent a ticket over to support@prefect.io. they should be getting back to us shortly
r
Hey @Hamza Kazmi and @Bob LeBoeuf!! We will have our support team respond in the support email thread to learn more about the issue.
gratitude thank you 1
b
@Rob Freedy https://meet.google.com/puk-cjgr-pxf If someone is available to join on your side, myself and two of our SRE engineers are on this call now investigating.
b
Hey team, marking this thread with a green check following our conversation from earlier
Notes from the meeting: It was noted that an upgrade to a newer version of Kubernetes (1.21) requires new brearer tokens (bound service account tokens) that were not utilized in the previous Kubernetes version that the flows were originally running with. The recommended troubleshooting steps were to: 1. Upgrade the agent to the latest 1.4.* version. 2. Delete the kubernetes pod and create a new one which produces an updated bearer token and cert.
Please feel free to add to/correct anything mentioned in this summary ^ 😄
3 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by