Guilherme Petris
11/21/2023, 2:54 PM@flow(name='pe_api_client_agreements_flow'
,task_runner=DaskTaskRunner())
def flow_run():
list_id = get_list_id()
returns = get_revisions_rate_limit.map(list_id)
I’ve try multiple ways on trying to limit the concurrency:
1. Task concurrency tags (@task(tags=[‘prefect_api_concurrency_limit’]) + Agent pool(Task Run Concurrency Limits)
a. Created and submitted tasks just passed by way over the limitation - Doesn’t work
2. Global concurrency limits (UI + code) - Limits are activated on the UI
a. It doesn’t limit the the submitted task created - Doesn’t work
It’s not well defined what needs to be done in what level and if should be done both on UI and code.
I’ve tested a bunch of test case scenarios and don’t see any of the provided tools working here. Anyone have thoughts on this? Any help is more than appreciated!Guilherme Petris
11/21/2023, 2:58 PMYing Ting Loo
11/22/2023, 10:09 AM