I'm seeing all my flows fail with `requests.except...
# ask-communityb
Brett Naul
06/29/2021, 6:24 PMI'm seeing all my flows fail with
requests.exceptions.RetryError: HTTPSConnectionPool(host='<http://api.prefect.io|api.prefect.io>', port=443): Max retries exceeded with url: /graphql (Caused by ResponseError('too many 429 error responses'))k
Kevin Kho
06/29/2021, 6:31 PM
Hey @Brett Naul, this is related to rate limiting. Are you running a lot of flows at the same time or do you have a lot of running flows? Will ask the team about this
b
Brett Naul
06/29/2021, 6:32 PMI didn't think so but it looks like there is kind of a lot, I'll look into why there's so many going
Brett Naul
06/29/2021, 6:50 PM@Kevin Kho can you give me some hints as to what kind of thing is getting rate limited? is it related to the # of active task runs, flow runs, ...?
k
Kevin Kho
06/29/2021, 6:52 PM
API calls. Each task state change is an API call. We’ll adjust the rate a bit on our end.
b
Brett Naul
07/19/2021, 10:04 PM@Kevin Kho we're seeing this error again so all of our flow runs are stuck. is there somewhere we can see this quota so this doesn't just happen out of the blue?
Brett Naul
07/19/2021, 10:27 PM^ this specific issue appears to have worked itself out but it would be nice to have some guidance around how to avoid hitting this in the future (I think we were locked out for a couple of hours)
k
Kevin Kho
07/20/2021, 3:56 PM
It’s my understanding that this got resolved via DM but let us know if you have any followups
3 Views