Can't quite figure this out from the docs: is there a way to limit the number of tasks in a map that are run in parallel?
j
josh
09/27/2019, 1:05 PM
I don’t believe there is currently a pure Prefect way to do it that is implemented yet. Here’s an issue someone raised which seems similar https://github.com/PrefectHQ/prefect/issues/1315 and there are some solutions provided on how you might accomplish task concurrency limiting.
Perhaps @Chris White has more information on this feature
z
Zachary Hughes
09/27/2019, 1:43 PM
Hi Tobias, we're currently developing task throttling as a Cloud feature! I don't have an exact release date right now, but can check with the team and get you a more concrete timeframe (believe it'll be ready in the very near future).
t
Tobias Schmidt
09/27/2019, 4:19 PM
Cool! My use case, in caseyou're interested, is a data load that ideally I'd want to do a few files at a time.
z
Zachary Hughes
09/27/2019, 5:39 PM
Awesome, we always appreciate more user input. 🙂 Working on getting an answer, will let you know as soon as I have one!