https://prefect.io logo
Title
g

Gregory Hunt

05/17/2023, 12:40 PM
How do streaming flows count for the API Rate Limits in Prefect Cloud?
āœ… 1
For example I would like to consumer message from a pub/sub in a streaming flow, does each message count towards my 2000/m and what happens in that flow if I get a spike in traffic.
n

Nate

05/17/2023, 1:37 PM
hi @Gregory Hunt - these rate limits should still apply. It just depends on how many API calls you make, and what variety they are - it shouldn't matter whether its a batch or streaming workflow
g

Gregory Hunt

05/17/2023, 1:39 PM
What is considered an API call?
If a steaming flow is running, is every time it gets a record to process considered an API call
n

Nate

05/17/2023, 1:46 PM
for example, when you load a block you have stored in prefect cloud, or you use
run_deployment
to hit
create_flow_run_from_deployment
or when your flow run / task run proposes a change in state that the orchestration engine in cloud must accept / reject .. all of these fall into the service-wide rate limit category. so this
If a steaming flow is running, is every time it gets a record to process considered an API call
depends on whether you're processing a record in a
task
or not, or creating an
Event
based on a record. If you're not doing either, then it doesn't contribute to your service wide Prefect Cloud API usage
you can turn on debug logs and run a flow if you want to get a sense for what api calls are happening behind the scenes
g

Gregory Hunt

05/17/2023, 1:48 PM
So for example, I want one streaming flow that is listening to a pub/sub and when it gets a message that has a file name in it, I have a flow with a few tasks that parse and process that file and then write it into a database
The flow would be constantly running and listening for messages on the subscription
@Nate I ran a flow in debug mode locally How do track which records are API calls
n

Nate

05/17/2023, 3:31 PM
ok turns out the docs were out of date, i just merged a PR to fix that, the service-wide rate limits are no longer enforced. so the only rate limits you should have to worry about against cloud are: • logs • flow and task run creation the numbers in the docs i linked above should still be accurate there isn't a direct empirical way I'm aware of to count all the API calls happening besides implementing some telemetry to count when certain endpoints are called from your code, but now that you're not constrained by service-wide rate limits, do you still need to?
g

Gregory Hunt

05/17/2023, 6:13 PM
I am just trying to understand if Prefect Cloud would work for my project or if it is better to role my own local server
šŸ‘ 1
@Nate So if task creation is an API cal is that tru that you can only create 2000 total tasks a minute, and if that is true what happens when a streaming flow exceeds that, are the tasks queued?
n

Nate

05/17/2023, 6:30 PM
If you exceed the task / flow run creation rate limit, we'll back-off and attempt to submit again some number of times (not sure off the top of my head how many) and eventually set the task/flow as Failed if it the rate limit is still exceeded
in general I'd say that Prefect Cloud task/flow creation rate limits are rarely a blocker for folks except at extreme scale, at which point it may make sense to have your own DB/server you can scale and configure - its hard to give a hard rule across the board though
happy to help estimate if you have more specific info on your use case