https://prefect.io logo
Channels
Powered by
j

Joshua Greenhalgh

07/13/2023, 1:58 PM
Is there a problem with prefect 2 cloud login at the moment?
👍 3
✅ 1
👀 1
r

Raphael Perez

07/13/2023, 1:59 PM
I believe so, it's down for us as well
j

Joshua Greenhalgh

07/13/2023, 1:59 PM
cool thanks
c

Christopher Boyd

07/13/2023, 1:59 PM
What issue are you seeing currently?
t

Trent Shapiro

07/13/2023, 2:00 PM
image.png
all workspaces gone
r

Raphael Perez

07/13/2023, 2:00 PM
Cannot login, both on UI and through the python cli
✅ 1
j

Joshua Greenhalgh

07/13/2023, 2:00 PM
Image below and console logs too
j

John Kang

07/13/2023, 2:00 PM
For us as well
c

Christopher Boyd

07/13/2023, 2:00 PM
Ok thank you - give me a minute
j

John Kang

07/13/2023, 2:00 PM
image.png
c

Christopher Boyd

07/13/2023, 2:00 PM
https://prefect.status.io/ - we are currently investigating performance and logins, standby
Thank you , investigating
Can you try again?
r

Raphael Perez

07/13/2023, 2:03 PM
It's back for me
✅ 1
z

Zachary Loertscher

07/13/2023, 2:04 PM
I can log in now, but flow runs page takes ~10s to load
c

Christopher Boyd

07/13/2023, 2:04 PM
Understood, we are still investigating performance issues
j

John Kang

07/13/2023, 2:19 PM
Flows aren't executing in the agent
image.png
👍 1
j

Joshua Greenhalgh

07/13/2023, 2:20 PM
All my deployments have dissapeared...
and flows and flow runs 😞
blocks....work pools eek
r

Raphael Perez

07/13/2023, 2:23 PM
The issue seems to be only on the prefect UI, if you run 
prefect flow ls
on an authenticated cli it correctly finds our flows
j

Joshua Greenhalgh

07/13/2023, 2:25 PM
Yeah for flows...but not for deployments 😞
r

Raphael Perez

07/13/2023, 2:28 PM
deployments you can find with 
prefect deployment ls
, doing a quick look around everything seems to be fine with our data, I can even trigger flows from the cli and our agent picks them up correctly, the issue is probably in the UI only
j

Joshua Greenhalgh

07/13/2023, 2:29 PM
I get a keyerror for deployments
Copy code
KeyError: UUID('570126de-6824-4a6b-9c81-24198ab8caee')
An exception occurred.
Anyway I assume this will get sorted and if not I can recreate the deployments anyway
c

Christopher Boyd

07/13/2023, 2:30 PM
This is likely only temporary as a result of the ongoing incident - your data / deployments should not be “gone”
j

Joshua Greenhalgh

07/13/2023, 2:32 PM
Yeah they look to be back now 😓
m

Michael Michael

07/13/2023, 2:57 PM
I think there is still a big delay with the api ? When I run a flow, the agent still shows me the following: "Submitting flow run id.... "but after that nothing happens for a while. After that the queue goes into the status unhealthy. No further logs are displayed in the agent. Does anyone else have this problem ?
c

Christopher Boyd

07/13/2023, 3:04 PM
You can enable debug logs on your agent to get more details - this issue should be resolved and back to full function, so I’d suspect your issue might be separate here?
m

Michael Michael

07/13/2023, 3:50 PM
Get a similar error as @John Kang posted above.

https://prefect-community.slack.com/files/U03F9TT4YG5/F05GQ5N7GK0/image.png▾

c

Christopher Boyd

07/13/2023, 3:51 PM
Is this for existing flow runs while we were experiencing issues, or new flow runs, or both ?
These look like just timeout errors , but it’s not clear what we are timing out on here , and in either scenario the api should be responsive at this time
j

John Kang

07/14/2023, 1:25 AM
@Christopher Boyd It looks like it is working now. Thanks for looking into it for us.
Powered by