Hi! Is there a way to concurrency limit a task running with

.map

? I've tried using CLI

prefect concurrency-limit

tagging the task but apperently there was no effect and the limit was not respected

Hm that limit should be respected. Can you share the output of

prefect version

?

Sure!

Version:             2.6.3
API version:         0.8.2
Python version:      3.8.13
Git commit:          9e7da96e
Built:               Tue, Oct 18, 2022 1:55 PM
OS/Arch:             linux/x86_64
Profile:             etl_dev
Server type:         cloud

I believe this should work normally with mapping. We made a performance change to concurrency limits in Cloud that caused a regression in some specific uses (I don’t quite remember which) and there’s a fix in the works (might have been deployed today). When did you encounter the unrespected limits? Have you tried the same code with the open source server?

I was developing it today. I haven't tried it with the open source server, I'll give it a try and comeback with the follow up. Thanks!

Great let me know!

Hello Michael! First all "sub-tasks" are created and then they are executed respecting the concurrency limit right? I tested with a small list and saw this behavior both in Cloud and Open Souce (local). In my production deployment the list has about 60.000 records and I'm thinkning maybe mapping is not the way to go. Just for context, the task make requests to an API using the record (ex.

GET - <https://base.url/code/{record}>

) that has a Rate Limit and upload the data to DB through another API (ex.

POST - <https://base.url/code/{record}>

) that also has Rate Limit, that's why I'm trying to use concurrency limit.

Hey @Esdras Lopes Nani, how are you doing? I'm curious to know what approach you finally adopted