We've started to get frequent "429 Too many Requests" errors in the past few days causing flows to fail and our work queue to go into an unhealthy state. We have not changed the number of flows or tasks that are running. has something changed on the prefect cloud side? Have the api limits changed? Are there other api request limits besides the per-minute limits

Hi @Walter Cavinaw, we're on the issue—I'll circle back with the team. Separately, we did adjust our rate limits to include all subroutes of

/flow_runs

and

/task_runs

yesterday. These limits rise as you go between the personal and organization tier. For more help dealing with the 429s, I'd encourage you to book a time with one of our product advocates, they'd be happy to help: https://calendly.com/prefect-experts/rubber-duck

Hi, thanks i noticed that you guys were adjusting the way rate limiting works. (Zanie Adkins mentioned it in the github issue last night). If i may provide some feedback to prefect and the team. I really enjoy the prefect product when it works so please consider this constructive criticism. I would suggest not changing rate limiting so dramatically (without notice - or at least I couldn't find any mention of it). It broke a lot of production workflows for us and we spent a significant amount of time trying to debug. Stability is a major concern for us when it comes to recent changes to prefect. I would also suggest differentiating the rate limits better. We are now paying 425 a month. It's not reasonable that rate limits for us are the same as someone on a free tier who is abusing Prefect Cloud. We can't justify moving to an org account and tripling how much we pay prefect with the issues we've experienced. The solution we ended up with is throttling task creation. The problem is that doing this...

func.map(list)

...creates all the tasks at once, immediately breaching the api limits. we've added a sleep between task creation but this just creates bad code tbh. just to reiterate, we love prefect, but you guys need to be much more careful with significant changes to the behaviour of the service, and create different api limit tiers or change how it works.

@Walter Cavinaw First of all, thank you so much for the detailed feedback. Getting this level of context into your use case is always a huge benefit for us. We'd love to chat with your team, mind booking a time in the Calendly I pasted above if you haven't already, or feel free to book a time with me directly: https://calendly.com/sarah-krasnik-prefect/30min?month=2023-05