<@U03FKQX1TH9> the same issue again with flows cra...
# ask-communitya
Alexey Stoletny
09/28/2022, 3:26 PM@Christopher Boyd the same issue again with flows crashing started to happen a few minutes ago:
Copy code
raise PrefectHTTPStatusError.from_httpx_error(exc) from exc.__cause__
prefect.exceptions.PrefectHTTPStatusError: Client error '403 Forbidden' for url '<https://api.prefect.cloud/api/accounts/43db7ccd-9f39-41f2-8989-000b28747858/workspaces/cedd89e9-9f12-421e-a17b-94045c976a2a/block_types/e91e7544-7ecd-4fa6-b6d7-a53068cb67fc>'
Response: {'detail': 'protected block types cannot be updated.'}
For more information check: <https://httpstatuses.com/403>✅ 1
k
Khuyen Tran
09/28/2022, 3:26 PMCan you link to the previous thread of this issue? Also to keep the channel clean, do you mind putting the related questions in one thread?
a
Alexey Stoletny
09/28/2022, 3:27 PMAlexey Stoletny
09/28/2022, 3:27 PMokay sounds good
Alexey Stoletny
09/28/2022, 3:29 PMtagging @Anna Geller and @Zanie who helped with the last issue as well, just in case
Alexey Stoletny
09/28/2022, 3:29 PMthis is affecting our production flow of data
z
Zanie
09/28/2022, 3:30 PM
Thanks for tagging! We’ll investigate.
🙏 1
Zanie
09/28/2022, 3:32 PM
This should be fixed again.
👍 2
Zanie
09/28/2022, 3:33 PM
We attempted to deploy a long term fix and had to roll it back for an unrelated reason and it caused a brief change here. We’ll be implementing some changes to avoid this in the future.
a
Alexey Stoletny
09/28/2022, 3:47 PMAppreciate it @Zanie
Alexey Stoletny
09/28/2022, 3:49 PMJust in case, we still have flows sometimes hanging for too long or failing because of a HTTP 500 on Prefect API, which is mostly mitigated via retries but still happens every few days; @Zanie I know you’re also investigating this, so figured I would ask
z
Zanie
09/28/2022, 3:50 PM
Which route are you getting a 500 on?
a
Alexey Stoletny
09/28/2022, 3:50 PMrandomly across a number, like getting a Secret block or even when the flow is being registered behind the scenes
Alexey Stoletny
09/28/2022, 3:50 PMthe flows sometimes fail with HTTP 500
Alexey Stoletny
09/28/2022, 3:51 PMChristopher suggested we do retries while this is investigated, and while that fixes it “most of the time” as it is intermittent, it still occasionally happens
Alexey Stoletny
09/28/2022, 3:52 PM@Zanie here’s a link to that: https://prefect-community.slack.com/archives/CL09KU1K7/p1663092920489649?thread_ts=1663087056.534729&cid=CL09KU1K7
Alexey Stoletny
09/28/2022, 3:52 PMsometimes it just times out and the agent becomes slow as it’s hanging in the API calls and flows become Late too
z
Zanie
09/28/2022, 3:53 PM
Thanks! I’m not working on Cloud performance, but I can forward this to the team that is.
Zanie
09/28/2022, 3:54 PM
I’m deep down this hole https://github.com/PrefectHQ/prefect/pull/6908 🙂
👍 2
a
Alexey Stoletny
09/28/2022, 4:40 PMUnderstood, thank you @Zanie