Sry to re-ask, but I wanna understand the meaning of each parameter for `Concurrency` :

Loup

10/12/2024, 9:47 AM

Sry to re-ask, but I wanna understand the meaning of each parameter for

Concurrency

Loup

10/12/2024, 9:48 AM

I have multiple API key, so TMDB have rate limit of 50 requests / seconde so I just dont wanna hit this rate limit. But what

Concurrency Limit

Active Slots

are made for ?

Nate

10/12/2024, 2:59 PM

hi @Loup - the docs on this are here! slot decay is just a way to set a time-based strategy where concurrency slots are released after the number of seconds you specify

Loup

10/13/2024, 9:47 AM

But if I just wanna protect the rate limit, like dont go above 50 requests per seconde, what should I set ? Because I dont care how many concurrency there is, just wanna slow down when rate limit is reached 😅

Loup

10/13/2024, 9:47 AM

Because if I well understood, if I increase the

Slot Decay per Second

its gonna be slower right ?

Loup

10/13/2024, 9:50 AM

Well Ive wrong i guess :

Copy code

To configure slot decay, set the slot_decay_per_second parameter when creating or updating a concurrency limit. This value determines how quickly slots refresh:

A higher value (for example, 5.0) means slots refresh quickly. Tasks can run more frequently, but with short pauses between them.
A lower value (for example, 0.1) means slots refresh slowly. Tasks run less frequently, with longer pauses between them.

Mhh Im kinda lost here 😭

Nate

10/13/2024, 5:14 PM

Because I dont care how many concurrency there is, just wanna slow down when rate limit is reached 😅

then Im not sure you need to worry about slot decay at all - just keep it at zero and use the

rate_limit

ctx manager and it sounds like what you describe is the default behavior

Loup

10/14/2024, 9:38 AM

But if I dont set a limit how can I know if the rate limit is reached. Sry Im kinda new to python I lack some notions

Jainish Savalia

12/06/2024, 8:27 AM

@Loup You want 50 RPS, which means you will need to allow 50 slots every second which is 50 concurrency limit and slot decay of 50

12 Views

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.