I have a quick question about Worker configuration. I see there is a
--limit
option. What does this do in practice? I have used the concurrency limit on work pools before, but I'm wondering what this achieves in addition to that.
✅ 1
j
Jake Kaplan
03/06/2024, 7:39 PM
hey, a this would keep the worker from submitting and monitoring more then
--limit
runs at a time.
Jake Kaplan
03/06/2024, 7:40 PM
you could have a work pool with concurrency limit 10 and if your worker had --limit 5, only 5 runs at a time would get picked up by that individual worker
j
Joey Allison
03/06/2024, 7:41 PM
so it's a kind of concurrency limit on the worker itself
Joey Allison
03/06/2024, 7:41 PM
where the work pool can also enforce a broader concurrency limit
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.