In one of my flows, I'm using a concurrency limit of 1 on a particular resource.
This is working, but I've noticed that the relevant task is only being executed once every 30 seconds or so. The rest of the time my flow is just sitting idle.
If I inspect the concurrency-limit via the CLI while my flow is running, "Active Task Run IDs" is empty most of the time.
Here is a
gist that demonstrates this. With a concurrency of N, you get batches of N tasks running with a 30 second delay between each batch. My case is just with N == 1.
Is there some kind of polling interval built into prefect, such that concurrency limits are only checked once every 30 seconds? If so, is there any way to reduce this delay? Is this behaviour documented anywhere?