https://prefect.io logo
Title
j

Jared Robbins

03/21/2022, 3:10 PM
Also, is it possible to limit concurrency at the deployment level? I am implementing a job that I don't want running twice at the same time. It seems tags in a DeploymentSpec don't work for concurrency limits. Right now I might move the tag to the main flow
k

Kevin Kho

03/21/2022, 3:26 PM
Tags should work on the deployment. I’ll try this in a bit
a

Anna Geller

03/21/2022, 3:48 PM
@Jared Robbins there are two levels of concurrency limits. #1 Currently, using the
prefect concurrency-limit
command, you can set task-level concurrency limits. 2# The alternative to task-level concurrency limits are work-queue concurrency limits which allow you to set limits on a deployment level - this seems to be exactly what you want! To create that, you can create a work-queue for a specific deployment UUID:
prefect work-queue create -d 'uuid' test_queue
this will return WORK_QUEUE_ID which you can use to set a limit for that to 1:
prefect work-queue set-concurrency-limit WORK_QUEUE_ID 1
j

Jared Robbins

03/21/2022, 3:51 PM
I suppose that will work... will need a work queue and agent for every job though
@Kevin Kho Was that possible? I think it would be a really useful feature. I'm sure I'm not the only one who doesn
't want the same job to run twice
k

Kevin Kho

03/21/2022, 4:40 PM
That I know for sure will be a built-in feature where the current run will just wait for the previous run to finish. It just hasn;’t been developed yet
j

Jared Robbins

03/21/2022, 5:20 PM
Dang. I'll keep an eye on the changelogs. will put this on the backburner for now